mlsanityData to 7 Oct 2026

Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications

arXiv 1606.03237 ↗First posted 10 Jun 2016cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

Facial expressions are an important way through which humans interact socially. Building a system capable of automatically recognizing facial expressions from images and video has been an intense field of study in recent years. Interpreting such expressions remains challenging and much research is needed about the way they relate to human affect. This paper presents a general overview of automatic RGB, 3D, thermal and multimodal facial expression analysis. We define a new taxonomy for the field, encompassing all steps from face detection to facial expression recognition, and describe and classify the state of the art methods accordingly. We also present the important datasets and the bench-marking of most influential methods. We conclude with a general discussion about trends, important questions and future lines of research.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 10; 2023: 6; 2024: 9; 2025: 6; 2026: 1.

45 papers.

Sort:

By area:

  1. 2022Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithmscited by 51
  2. 2023Micron-BERT: BERT-based Facial Micro-Expression Recognitioncited by 43
  3. 2023Prompting Visual-Language Models for Dynamic Facial Expression Recognitioncited by 43
  4. 2021Global-Local Attention for Emotion Recognitioncited by 20
  5. 2023Neuromorphic Event-based Facial Expression Recognitioncited by 18
  6. 2021Graph-based Facial Affect Analysis: A Reviewcited by 14
  7. 2021Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognitioncited by 12
  8. 2021Towards Unbiased Visual Emotion Recognition via Causal Interventioncited by 11
  9. 2021CapsField: Light Field-based Face and Expression Recognition in the Wild using Capsule Routingcited by 9
  10. 2023A Comprehensive Survey on Affective Computing; Challenges, Trends, Applications, and Future Directionscited by 7
  11. 20233D Face Reconstruction: the Road to Forensicscited by 7
  12. 2024Neuromorphic Facial Analysis with Cross-Modal Supervisioncited by 7
33 more
  1. 2022An EEG-Based Multi-Modal Emotion Database with Both Posed and Authentic Facial Actions for Emotion Analysiscited by 6
  2. 2021DeepFN: Towards Generalizable Facial Action Unit Recognition with Deep Face Normalizationcited by 5
  3. 2022When Facial Expression Recognition Meets Few-Shot Learning: A Joint and Alternate Learning Frameworkcited by 5
  4. 2024Two in One Go: Single-stage Emotion Recognition with Decoupled Subject-context Transformercited by 5
  5. 2024Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Surveycited by 5
  6. 2021Interpreting Depression From Question-wise Long-term Video Recording of SDS Evaluationcited by 4
  7. 2021Transfer-based adaptive tree for multimodal sentiment analysis based on user latent aspectscited by 4
  8. 2021Critically examining the Domain Generalizability of Facial Expression Recognition modelscited by 4
  9. 2022AU-Aware Vision Transformers for Biased Facial Expression Recognitioncited by 4
  10. 2024Multi-Scale Spatio-Temporal Graph Convolutional Network for Facial Expression Spottingcited by 4
  11. 2024HDRT: A Large-Scale Dataset for Infrared-Guided HDR Imagingcited by 4
  12. 2022Adaptively Enhancing Facial Expression Crucial Regions via Local Non-Local Joint Networkcited by 3
  13. 2023Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coachcited by 3
  14. 2024SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spottingcited by 3
  15. 2021Label Distribution Amendment with Emotional Semantic Correlations for Facial Expression Recognitioncited by 2
  16. 2022VISTANet: VIsual Spoken Textual Additive Net for Interpretable Multimodal Emotion Recognitioncited by 2
  17. 2024Emotion Recognition from the perspective of Activity Recognitioncited by 2
  18. 2024Spatio-temporal Transformers for Action Unit Classification with Event Camerascited by 2
  19. 2021Towards Fine-grained 3D Face Dense Registration: An Optimal Dividing and Diffusing Methodcited by 1
  20. 2022Transferring Dual Stochastic Graph Convolutional Network for Facial Micro-expression Recognitioncited by 1
  21. 2025Contrastive Language-Image Learning with Augmented Textual Prompts for 3D/4D FER Using Vision-Language Modelcited by 1
  22. 2025Unsupervised Multiview Contrastive Language-Image Joint Learning with Pseudo-Labeled Prompts Via Vision-Language Model for 3D/4D Facial Expression Recognitioncited by 1
  23. 2025A Perception CNN for Facial Expression Recognitioncited by 1
  24. 2021Domain Adaptation for Facial Expression Classifier via Domain Discrimination and Gradient Reversalcited by 0
  25. 2021Facial Emotion Recognition using Deep Residual Networks in Real-World Environmentscited by 0
  26. 20222D+3D facial expression recognition via embedded tensor manifold regularizationcited by 0
  27. 2022A Novel Fully Annotated Thermal Infrared Face Dataset: Recorded in Various Environment Conditions and Distances From The Cameracited by 0
  28. 2022AU-Supervised Convolutional Vision Transformers for Synthetic Facial Expression Recognitioncited by 0
  29. 2024AI Competitions and Benchmarks: Dataset Developmentcited by 0
  30. 2025Towards Fair and Robust Face Parsing for Generative AI: A Multi-Objective Approachcited by 0
  31. 2025Self-Supervised Multi-View Representation Learning using Vision-Language Model for 3D/4D Facial Expression Recognitioncited by 0
  32. 2025Facial Emotion Learning with Text-Guided Multiview Fusion via Vision-Language Model for 3D/4D Facial Expression Recognitioncited by 0
  33. 2026Beyond Universality: The GCC-FER Dataset and Culture-Aware Adaptation for Dynamic Facial Expression Recognitioncited by 0