# Computer Vision Estimation of Emotion Reaction Intensity in the Wild

arXiv: https://arxiv.org/abs/2303.10741 · HTML: https://mlsanity.com/paper/2303.10741/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 74 references that are in the map.
- First posted on arXiv 19 Mar 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace](https://mlsanity.com/paper/1910.04855/) (2019)
- [Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond](https://mlsanity.com/paper/1804.10938/) (2018)
- [Analysing Affective Behavior in the First ABAW 2020 Competition](https://mlsanity.com/paper/2001.11409/) (2020)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study](https://mlsanity.com/paper/2105.03790/) (2021)
- [Analysing Affective Behavior in the second ABAW2 Competition](https://mlsanity.com/paper/2106.15318/) (2021)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network](https://mlsanity.com/paper/1910.11111/) (2019)
- [ABAW: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.01138/) (2022)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges](https://mlsanity.com/paper/2303.01498/) (2023)
- [Transformer-based Multimodal Information Fusion for Facial Expression Analysis](https://mlsanity.com/paper/2203.12367/) (2022)
- [The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress](https://mlsanity.com/paper/2207.05691/) (2022)
- [Continuous Emotion Recognition using Visual-audio-linguistic information: A Technical Report for ABAW3](https://mlsanity.com/paper/2203.13031/) (2022)
- [Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition](https://mlsanity.com/paper/2107.03708/) (2021)
- [Coarse-to-Fine Cascaded Networks with Smooth Predicting for Video Facial Expression Recognition](https://mlsanity.com/paper/2203.13052/) (2022)
- [Frame-level Prediction of Facial Expressions, Valence, Arousal and Action Units for Mobile Devices](https://mlsanity.com/paper/2203.13436/) (2022)
- [Activity Recognition with Moving Cameras and Few Training Examples: Applications for Detection of Autism-Related Headbanging](https://mlsanity.com/paper/2101.03478/) (2021)
- [Multi-modal Emotion Estimation for in-the-wild Videos](https://mlsanity.com/paper/2203.13032/) (2022)
- [Automatic Facial Expression Recognition Using Features of Salient Facial Patches](https://mlsanity.com/paper/1505.04026/) (2015)
- [Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems](https://mlsanity.com/paper/2002.06581/) (2020)
- [Hybrid Multimodal Feature Extraction, Mining and Fusion for Sentiment Analysis](https://mlsanity.com/paper/2208.03051/) (2022)
- [A Review of and Roadmap for Data Science and Machine Learning for the Neuropsychiatric Phenotype of Autism](https://mlsanity.com/paper/2303.03577/) (2023)
- [Classification of Abnormal Hand Movement for Aiding in Autism Detection: Machine Learning Study](https://mlsanity.com/paper/2108.07917/) (2021)
- [Facial Action Unit Recognition With Multi-models Ensembling](https://mlsanity.com/paper/2203.13046/) (2022)
- [Training Affective Computer Vision Models by Crowdsourcing Soft-Target Labels](https://mlsanity.com/paper/2101.03477/) (2021)
- [Training and Profiling a Pediatric Emotion Recognition Classifier on Mobile Devices](https://mlsanity.com/paper/2108.11754/) (2021)

## Built on by (most cited first, 3 of 3)

- [FIESTA: Fisher Information-based Efficient Selective Test-time Adaptation](https://mlsanity.com/paper/2503.23257/) (2025)
- [Fine-tuning a multimodal large language model for clinician-grade autism behavioral scoring from short home videos](https://mlsanity.com/paper/2606.27484/) (2026)
- [Facial Expression Recognition in the Deep Learning Era: A Systematic Multi-Criteria Review of Methods, Models, Datasets, Performance, Challenges, and Future Research Directions](https://mlsanity.com/paper/2606.08612/) (2026)

## Abstract (arXiv)

Emotions play an essential role in human communication. Developing computer vision models for automatic recognition of emotion expression can aid in a variety of domains, including robotics, digital behavioral healthcare, and media analytics. There are three types of emotional representations which are traditionally modeled in affective computing research: Action Units, Valence Arousal (VA), and Categorical Emotions. As part of an effort to move beyond these representations towards more fine-grained labels, we describe our submission to the newly introduced Emotional Reaction Intensity (ERI) Estimation challenge in the 5th competition for Affective Behavior Analysis in-the-Wild (ABAW). We developed four deep neural networks trained in the visual domain and a multimodal model trained with both visual and audio features to predict emotion reaction intensity. Our best performing model on the Hume-Reaction dataset achieved an average Pearson correlation coefficient of 0.4080 on the test set using a pre-trained ResNet50 model. This work provides a first step towards the development of production-grade models which predict emotion reaction intensities rather than discrete emotion categories.