mlsanityData to 14 Oct 2026

OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysis

arXiv 2506.02891 ↗First posted 3 Jun 2025cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

In recent years, there has been increasing interest in automatic facial behavior analysis systems from computing communities such as vision, multimodal interaction, robotics, and affective computing. Building upon the widespread utility of prior open-source facial analysis systems, we introduce OpenFace 3.0, an open-source toolkit capable of facial landmark detection, facial action unit detection, eye-gaze estimation, and facial emotion recognition. OpenFace 3.0 contributes a lightweight unified model for facial analysis, trained with a multi-task architecture across diverse populations, head poses, lighting conditions, video resolutions, and facial analysis tasks. By leveraging the benefits of parameter sharing through a unified model and training paradigm, OpenFace 3.0 exhibits improvements in prediction performance, inference speed, and memory efficiency over similar toolkits and rivals state-of-the-art models. OpenFace 3.0 can be installed and run with a single line of code and operate in real-time without specialized hardware. OpenFace 3.0 code for training models and running the system is freely available for research purposes and supports contributions from the community.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 44 of its 92 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

44 of 92 references.

References · 44

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  6. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  8. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  9. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  10. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  11. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  12. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 492
32 more
  1. 2019Gaze360: Physically Unconstrained Gaze Estimation in the Wildcited by 174
  2. 2016Eye Tracking for Everyonecited by 155
  3. 2015Appearance-Based Gaze Estimation in the Wildcited by 153
  4. 2018Look at Boundary: A Boundary-Aware Face Alignment Algorithmcited by 138
  5. 2016It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimationcited by 123
  6. 2021Graph Stacked Hourglass Networks for 3D Human Pose Estimationcited by 95
  7. 2019AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognitioncited by 87
  8. 2021TransFER: Learning Relation-aware Facial Expression Representations with Transformerscited by 85
  9. 2022Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognitioncited by 82
  10. 2021Gaze Estimation using Transformercited by 68
  11. 2022L2CS-Net: Fine-Grained Gaze Estimation in Unconstrained Environmentscited by 65
  12. 2018Facial Landmark Detection: a Literature Surveycited by 63
  13. 2022POSTER: A Pyramid Cross-Fusion Transformer Network for Facial Expression Recognitioncited by 49
  14. 2022Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learningcited by 47
  15. 2023STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detectioncited by 43
  16. 2023SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimationcited by 39
  17. 2018Deep Structure Inference Network for Facial Action Unit Recognitioncited by 38
  18. 2024FaceXFormer: A Unified Transformer for Facial Analysiscited by 34
  19. 2021Py-Feat: Python Facial Expression Analysis Toolboxcited by 33
  20. 2022High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learningcited by 31
  21. 2024AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectorscited by 29
  22. 2022Shape Preserving Facial Landmarks with Graph Attention Networkscited by 23
  23. 2024Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questionscited by 22
  24. 2023LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysiscited by 18
  25. 2024Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classificationcited by 16
  26. 2021HIH: Towards More Accurate Face Alignment via Heatmap in Heatmapcited by 15
  27. 2023End-to-end Video Gaze Estimation via Capturing Head-face-eye Spatial-temporal Interaction Contextcited by 14
  28. 2015Facial Landmark Detection with Tweaked Convolutional Neural Networkscited by 13
  29. 2024Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognitioncited by 11
  30. 2024Norface: Improving Facial Expression Analysis by Identity Normalizationcited by 10
  31. 2022AFFDEX 2.0: A Real-Time Facial Expression Analysis Toolkitcited by 5
  32. 2024Fiducial Focus Augmentation for Facial Landmark Detectioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026TAG: Thinking with Action Unit Grounding for Facial Expression Recognitioncited by 2
  2. 2026GLEaN: A Text-to-image Bias Detection Approach for Public Comprehensioncited by 0
  3. 2026Human-Inspired Context-Selective Multimodal Memory for Social Robotscited by 0
  4. 2026ThinkDeception: A Progressive Reinforcement Learning Framework for Interpretable Multimodal Deception Detectioncited by 0
  5. 2026Occlusion-Robust Multimodal Emotion Recognition in VR via Fusion of Facial Images and EMGcited by 0