mlsanityData to 7 Oct 2026

Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition

arXiv 2103.13372 ↗First posted 24 Mar 2021cs.CV cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Temporal context is key to the recognition of expressions of emotion. Existing methods, that rely on recurrent or self-attention models to enforce temporal consistency, work on the feature level, ignoring the task-specific temporal dependencies, and fail to model context uncertainty. To alleviate these issues, we build upon the framework of Neural Processes to propose a method for apparent emotion recognition with three key novel components: (a) probabilistic contextual representation with a global latent variable model; (b) temporal context modelling using task-specific predictions in addition to features; and (c) smart temporal context selection. We validate our approach on four databases, two for Valence and Arousal estimation (SEWA and AffWild2), and two for Action Unit intensity estimation (DISFA and BP4D). Results show a consistent improvement over a series of strong baselines as well as over state-of-the-art methods.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 35 of its 76 references that are in the map. First posted on arXiv 24 Mar 2021.

Builds on

35 of 76 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  6. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  7. 2018Conditional Neural Processescited by 421
  8. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  9. 2018Neural Processescited by 307
  10. 2019Attentive Neural Processescited by 280
  11. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  12. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
23 more
  1. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  2. 2019RetinaFace: Single-stage Dense Face Localisation in the Wildcited by 164
  3. 2019Convolutional Conditional Neural Processescited by 139
  4. 2019Exact Gaussian Processes on a Million Data Pointscited by 118
  5. 2019AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognitioncited by 87
  6. 2018Aff-Wild2: Extending the Aff-Wild Database for Affect Recognitioncited by 80
  7. 2020Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processescited by 74
  8. 2019SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wildcited by 62
  9. 2019Sequential Neural Processescited by 55
  10. 2020Two-Stream Aural-Visual Affect Analysis in the Wildcited by 51
  11. 2019Factorized Higher-Order CNNs with an Application to Spatio-Temporal Emotion Estimationcited by 40
  12. 2020Bootstrapping Neural Processescited by 40
  13. 2019Exploiting multi-CNN features in CNN-RNN based Dimensional Emotion Recognition on the OMG in-the-wild Datasetcited by 39
  14. 2020Multitask Emotion Recognition with Incomplete Labelscited by 39
  15. 2018A Multi-Task Learning & Generation Framework: Valence-Arousal, Action Units & Primary Expressionscited by 25
  16. 2019Modeling emotion in complex stories: the Stanford Emotional Narratives Datasetcited by 20
  17. 2017Deep Structured Learning for Facial Action Unit Intensity Estimationcited by 16
  18. 2019Deep Structured Mixtures of Gaussian Processescited by 16
  19. 2019The Ambiguous World of Emotion Representationcited by 15
  20. 2017DeepCoder: Semi-parametric Variational Autoencoders for Automatic Facial Action Codingcited by 9
  21. 2020A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimationcited by 9
  22. 2015Variable-state Latent Conditional Random Fields for Facial Expression Recognition and Action Unit Detectioncited by 3
  23. 2016Variational Gaussian Process Auto-Encoder for Ordinal Prediction of Facial Action Unitscited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 6; 2023: 3; 2024: 4; 2025: 1.

15 papers.

Sort:

  1. 2024Benchmarking Micro-action Recognition: Dataset, Methods, and Applicationscited by 48
  2. 2022COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognitioncited by 22
  3. 2024MMAD: Multi-label Micro-Action Detection in Videoscited by 20
  4. 2022Facial Action Unit Detection and Intensity Estimation from Self-supervised Representationcited by 14
  5. 2023FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Featurescited by 12
  6. 2022Optimal Transport-based Identity Matching for Identity-invariant Facial Expression Recognitioncited by 8
  7. 2021Technical Report for Valence-Arousal Estimation in ABAW2 Challengecited by 6
  8. 2022Are 3D Face Shapes Expressive Enough for Recognising Continuous Emotions and Action Unit Intensities?cited by 4
  9. 2023In the Blink of an Eye: Event-based Emotion Recognitioncited by 4
  10. 2024Apprenticeship-Inspired Elegance: Synergistic Knowledge Distillation Empowers Spiking Neural Networks for Efficient Single-Eye Emotion Recognitioncited by 3
  11. 2022SS-VAERR: Self-Supervised Apparent Emotional Reaction Recognition from Videocited by 2
  12. 2024AUEditNet: Dual-Branch Facial Action Unit Intensity Manipulation with Implicit Disentanglementcited by 1
3 more
  1. 2022Contextual Intelligent Decisions: Expert Moderation of Machine Outputs for Fair Assessment of Commercial Drivingcited by 0
  2. 2023Occlusion Aware Student Emotion Recognition based on Facial Action Unit Detectioncited by 0
  3. 2025xTrace: A Facial Expressive Behaviour Analysis Tool for Continuous Affect Recognitioncited by 0