mlsanityData to 7 Oct 2026

Graph-based Facial Affect Analysis: A Review

arXiv 2103.15599 ↗First posted 29 Mar 2021cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

As one of the most important affective signals, facial affect analysis (FAA) is essential for developing human-computer interaction systems. Early methods focus on extracting appearance and geometry features associated with human affects while ignoring the latent semantic information among individual facial changes, leading to limited performance and generalization. Recent work attempts to establish a graph-based representation to model these semantic relationships and develop frameworks to leverage them for various FAA tasks. This paper provides a comprehensive review of graph-based FAA, including the evolution of algorithms and their applications. First, the FAA background knowledge is introduced, especially on the role of the graph. We then discuss approaches widely used for graph-based affective representation in literature and show a trend towards graph construction. For the relational reasoning in graph-based FAA, existing studies are categorized according to their non-deep or deep learning methods, emphasizing the latest graph neural networks. Performance comparisons of the state-of-the-art graph-based FAA methods are also summarized. Finally, we discuss the challenges and potential directions. As far as we know, this is the first survey of graph-based FAA methods. Our findings can serve as a reference for future research in this field.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 57 of its 236 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

57 of 236 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2017Graph Attention Networkscited by 8,854
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  8. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  9. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  10. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  11. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  12. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
38 more
  1. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  2. 2015Gated Graph Sequence Neural Networkscited by 792
  3. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  4. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  5. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  6. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  7. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  8. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
  9. 2018Deep Face Recognition: A Surveycited by 275
  10. 2018Deep Facial Expression Recognition: A Surveycited by 269
  11. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  12. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  13. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  14. 2019Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognitioncited by 127
  15. 2016HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognitioncited by 103
  16. 2019Context-Aware Emotion Recognition Networkscited by 99
  17. 2018Bayesian graph convolutional neural networks for semi-supervised classificationcited by 88
  18. 2019AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognitioncited by 87
  19. 2020Context Based Emotion Recognition using EMOTIC Datasetcited by 73
  20. 2016From Facial Expression Recognition to Interpersonal Relation Predictioncited by 53
  21. 2019Semantic Relationships Guided Representation Learning for Facial Action Unit Recognitioncited by 53
  22. 2016Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applicationscited by 45
  23. 2015Going Deeper in Facial Expression Recognition using Deep Neural Networkscited by 43
  24. 2018Deep Structure Inference Network for Facial Action Unit Recognitioncited by 38
  25. 2021Supervision by Registration and Triangulation for Landmark Detectioncited by 30
  26. 2017EAC-Net: A Region-based Deep Enhancing and Cropping Approach for Facial Action Unit Detectioncited by 26
  27. 2019Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognitioncited by 26
  28. 2019Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detectioncited by 25
  29. 2019Deep Learning for Human Affect Recognition: Insights and New Developmentscited by 23
  30. 2020MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Networkcited by 21
  31. 2020Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolutioncited by 21
  32. 2018A Survey of Automatic Facial Micro-expression Analysis: Databases, Methods and Challengescited by 16
  33. 2019The Ambiguous World of Emotion Representationcited by 15
  34. 2014Facial Feature Point Detection: A Comprehensive Surveycited by 13
  35. 2019CenterFace: Joint Face Detection and Alignment Using Face as Pointcited by 13
  36. 2020Video-based Facial Expression Recognition using Graph Convolutional Networkscited by 13
  37. 2020Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognitioncited by 12
  38. 2017FERA 2017 - Addressing Head Pose in the Third Facial Expression Recognition and Analysis Challengecited by 9

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 3; 2024: 2; 2025: 3; 2026: 2.

14 papers.

Sort:

  1. 2023Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognitioncited by 17
  2. 2023Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detectioncited by 14
  3. 2023Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotioncited by 12
  4. 2022Uncertain Facial Expression Recognition via Multi-task Assisted Correctioncited by 10
  5. 2022Uncertain Label Correction via Auxiliary Action Unit Graphs for Facial Expression Recognitioncited by 8
  6. 2024Engagement Measurement Based on Facial Landmarks and Spatial-Temporal Graph Convolutional Networkscited by 7
  7. 2022Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognitioncited by 6
  8. 2024Authentic Emotion Mapping: Benchmarking Facial Expressions in Real Newscited by 1
  9. 2022Exploring Interactions and Regulations in Collaborative Learning: An Interdisciplinary Multimodal Datasetcited by 0
  10. 2025GraphAU-Pain: Graph-based Action Unit Representation for Pain Intensity Estimationcited by 0
  11. 2025FG 2025 TrustFAA: the First Workshop on Towards Trustworthy Facial Affect Analysis: Advancing Insights of Fairness, Explainability, and Safety (TrustFAA)cited by 0
  12. 2025OPEN: A Benchmark Dataset and Baseline for Older Adult Patient Engagement Recognition in Virtual Rehabilitation Learning Environmentscited by 0
2 more
  1. 2026Facial Affect Analysis for Service-Oriented Systems: Advances, Challenges, and Future Visionscited by 0
  2. 2026MoTop: Motion-Topological Model For Micro AU Detectioncited by 0