mlsanityData to 14 Oct 2026

Computer Vision Estimation of Emotion Reaction Intensity in the Wild

arXiv 2303.10741 ↗First posted 19 Mar 2023cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Emotions play an essential role in human communication. Developing computer vision models for automatic recognition of emotion expression can aid in a variety of domains, including robotics, digital behavioral healthcare, and media analytics. There are three types of emotional representations which are traditionally modeled in affective computing research: Action Units, Valence Arousal (VA), and Categorical Emotions. As part of an effort to move beyond these representations towards more fine-grained labels, we describe our submission to the newly introduced Emotional Reaction Intensity (ERI) Estimation challenge in the 5th competition for Affective Behavior Analysis in-the-Wild (ABAW). We developed four deep neural networks trained in the visual domain and a multimodal model trained with both visual and audio features to predict emotion reaction intensity. Our best performing model on the Hume-Reaction dataset achieved an average Pearson correlation coefficient of 0.4080 on the test set using a pre-trained ResNet50 model. This work provides a first step towards the development of production-grade models which predict emotion reaction intensities rather than discrete emotion categories.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 31 of its 74 references that are in the map. First posted on arXiv 19 Mar 2023.

Builds on

31 of 74 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 492
  5. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  6. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  7. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  8. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  9. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  10. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  11. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  12. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
19 more
  1. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  2. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  3. 2022Transformer-based Multimodal Information Fusion for Facial Expression Analysiscited by 49
  4. 2022The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stresscited by 23
  5. 2022Continuous Emotion Recognition using Visual-audio-linguistic information: A Technical Report for ABAW3cited by 20
  6. 2021Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competitioncited by 18
  7. 2022Coarse-to-Fine Cascaded Networks with Smooth Predicting for Video Facial Expression Recognitioncited by 17
  8. 2022Frame-level Prediction of Facial Expressions, Valence, Arousal and Action Units for Mobile Devicescited by 16
  9. 2021Activity Recognition with Moving Cameras and Few Training Examples: Applications for Detection of Autism-Related Headbangingcited by 15
  10. 2022Multi-modal Emotion Estimation for in-the-wild Videoscited by 15
  11. 2015Automatic Facial Expression Recognition Using Features of Salient Facial Patchescited by 13
  12. 2020Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systemscited by 12
  13. 2022Hybrid Multimodal Feature Extraction, Mining and Fusion for Sentiment Analysiscited by 12
  14. 2023A Review of and Roadmap for Data Science and Machine Learning for the Neuropsychiatric Phenotype of Autismcited by 11
  15. 2021Classification of Abnormal Hand Movement for Aiding in Autism Detection: Machine Learning Studycited by 10
  16. 2022Facial Action Unit Recognition With Multi-models Ensemblingcited by 10
  17. 2021Training Affective Computer Vision Models by Crowdsourcing Soft-Target Labelscited by 8
  18. 2021Training and Profiling a Pediatric Emotion Recognition Classifier on Mobile Devicescited by 5
  19. 2020Designing a Holistic At-Home Learning Aid for Autismcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025FIESTA: Fisher Information-based Efficient Selective Test-time Adaptationcited by 2
  2. 2026Fine-tuning a multimodal large language model for clinician-grade autism behavioral scoring from short home videoscited by 1
  3. 2026Facial Expression Recognition in the Deep Learning Era: A Systematic Multi-Criteria Review of Methods, Models, Datasets, Performance, Challenges, and Future Research Directionscited by 0