Education
- Ph.D. in Computer Science, University of Delaware. 2019 - Current
Advisor: Roghayeh Barmaki - M.S. in Computer Science, Michigan State University, 2019
- M.S. in Electrical Engineering, Northeastern University, 2016
- B.E. in Electrical Engineering, Harbin Institute of Technology, 2015
Skills
- Programming Languages: Python, Java, C++, MATLAB, SQL, R, Bash
- Libraries/Tools: PyTorch, TensorFlow, Keras, Sklearn, NumPy, SciPy, OpenCV, Pandas, OpenCL, AWS, Git, FFmpeg, LATEX
Work Experience
- Research Intern, Futurewei Technologies, Inc., Jun 2023 – Present
- Developed a deep learning-based system for precise visual-audio facial expression retargeting, prioritizing accurate lip synchronization and realistic talking head/avatar generation
- Research Assistant, University of Delaware, Human-Computer Interaction Lab, Jun 2019 – Present
- Developed systematic plans for dataset collection, problem definition, and solution development in multidisciplinary research using cutting-edge deep learning techniques for video understanding and human behavior analysis, with a focus on action quality assessment and intervention evaluation
- Established a large-scale, multimodal benchmark dataset comprising over 244K frames for behavior evaluation in autism research. Integrated diverse modalities including optical flow, skeleton, gaze, and clinician evaluation scores, one of the first publicly accessible datasets in the field
- Built deep learning methods that exclusively utilize privacy-preserving data, overcoming critical data-sharing challenges in sensitive research fields. Applied knowledge distillation to further enhance model performance and leverage video information without compromising privacy
- Designed a novel skeleton-based, uncertainty-aware Transformer Network to assess action quality while being robust to pose detection failures, demonstrating state-of-the-art performance on synchrony score prediction with a decrease in mean square error by 53.8% on the TASD-2 dataset
- Proposed an innovative two-stage affect states recognition framework that integrates facial expressions and acoustic cues, effectively leveraging the synergy between human expertise and machine intelligence
- Research Intern, Samsung Research America (Digital Health), Jan 2023 – Mar 2023
- Developed an innovative evaluation metric to assess facial alignment in remote vital sign detection, one of the first works to provide comprehensive guidelines for optimal face detector selection in the field
- Conducted experiments validating a 12.5% reduction in second-level mean absolute error and a 3.5% enhancement in accuracy for remote heart rate estimation by leveraging the superior face detector recommended by our metric
- Paper accepted for oral presentation at the International Conference on Body Sensor Networks – Sensors and Systems for Digital Health (IEEE BSN 2023)
- Machine Learning Engineer, GENISAMA LLC, Jan 2017 – Aug 201
- Developed optimized GPU kernels using OpenCL to accelerate the training and inference time of neural networks
- Achieved an average 4x speedup compared to CPU runtime, leading to substantial efficiency improvements
- Deployed the application on both PC and Android platforms, ensuring widespread accessibility and usability
- Performed debugging, testing, and version updates to ensure smooth operation and integration of new features
Publications
- Roghayeh,B., Li, J., et al., Towards Anatomy Education with Generative AI-based Virtual Assistants in Immersive Virtual Reality Environments, under review
- Sydney S., Vuthea,C., Li, J., et al., Visual Feedback and Guided Balance Training in the Immersive Virtual Reality Environment for Lower Extremity Rehabilitation, under review
- Li, J., et al., Advancements in Face Misalignment Evaluation for Contact-less Vital Sign Detection, International Conference on Body Sensor Networks: Sensor and Systems for Digital Health, 2023
- Li, J., et al., MMASD: A Multimodal Dataset for Autism Intervention Analysis, International Conference on Multimodal Interaction, 2023
- Guo, Z., Li, J., et al., Social Visual Behavior Analytics for Autism Therapy of Children Based on Automated Mutual Gaze Detection, IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies, 2023
- Li, J., et al., Pose Uncertainty Aware Movement Synchrony Estimation via Spatial-Temporal Graph Transformer, International Conference on Multimodal Interaction, 2022 (Oral)
- Li, J., et al., Dyadic Movement Synchrony Estimation Under Privacy-preserving Conditions, International Conference on Pattern Recognition, 2022
- Li, J., et al., Improving the Movement Synchrony Estimation with Action Quality Assessment in Children Play Therapy, International Conference on Multimodal Interaction, 2021
- Li, J., et al., A Two-stage Multi-modal Affect Analysis Framework for Children with Autism Spectrum Disorder, AAAI Workshop on Affective Content Analysis, 2021
- Bi, X., Li, J., et al., Constrained Optimization based on ε Constrained Biogeography-Based Optimization with Dynamic Migration, Journal of Computer Research and Development, 2014
Professional Honors And Awards
- CIS Distinguished Graduate Student Award, University of Delaware, 2020 – 2022
- Graduate Block Stipend Fellowship, University of Delaware, 2021
- Graduate Office Fellowship, Michigan State University, 2017
- Meritorious Winner at Science and Technology Innovation Contest, Harbin Institute of Technology, 2014
Patent
- Bluetooth module based access control system and access control method thereof
- An innovative access control method of controlling a mechanical lock via MCU and mobile phone
- Publication number: CN103295303B
Teaching Experience
- Teaching Assistant, University of Delaware, Sep 2019 – Present
- CISC 474 Advanced Web Technologies
- CISC 681 Artificial Intelligence
- CISC 682 Human-Computer Interaction
- Teaching Assistant, Michigan State University, Sep 2017 – May 2018
Membership
- Association for the Advancement of Artificial Intelligence, 2021 – Present
- Association for Computing Machinery, 2021 – Present
