# Context De-confounded Emotion Recognition

arXiv: https://arxiv.org/abs/2303.11921 · HTML: https://mlsanity.com/paper/2303.11921/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 62 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Unbiased Scene Graph Generation from Biased Training](https://mlsanity.com/paper/2002.11949/) (2020)
- [Visual Commonsense R-CNN](https://mlsanity.com/paper/2002.12204/) (2020)
- [Context-Aware Emotion Recognition Networks](https://mlsanity.com/paper/1908.05913/) (2019)
- [Two Causal Principles for Improving Visual Dialog](https://mlsanity.com/paper/1911.10496/) (2019)
- [Context Based Emotion Recognition using EMOTIC Dataset](https://mlsanity.com/paper/2003.13401/) (2020)
- [iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis](https://mlsanity.com/paper/2107.00285/) (2021)
- [Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Models](https://mlsanity.com/paper/2302.05087/) (2023)
- [EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle](https://mlsanity.com/paper/2003.06692/) (2020)
- [Towards Practical Certifiable Patch Defense with Vision Transformer](https://mlsanity.com/paper/2203.08519/) (2022)
- [Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias](https://mlsanity.com/paper/1808.02212/) (2018)
- [STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits](https://mlsanity.com/paper/1910.12906/) (2019)
- [TSA-Net: Tube Self-Attention Network for Action Quality Assessment](https://mlsanity.com/paper/2201.03746/) (2022)
- [Counterfactually-Augmented SNLI Training Data Does Not Yield Better Generalization Than Unaugmented Data](https://mlsanity.com/paper/2010.04762/) (2020)
- [De-biasing Distantly Supervised Named Entity Recognition via Causal Intervention](https://mlsanity.com/paper/2106.09233/) (2021)
- [Learning Appearance-motion Normality for Video Anomaly Detection](https://mlsanity.com/paper/2207.13361/) (2022)
- [A Survey of Video-based Action Quality Assessment](https://mlsanity.com/paper/2204.09271/) (2022)
- [CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Space](https://mlsanity.com/paper/2207.07869/) (2022)
- [Causal Intervention for Subject-Deconfounded Facial Action Unit Recognition](https://mlsanity.com/paper/2204.07935/) (2022)
- [Adversarial Contrastive Distillation with Adaptive Denoising](https://mlsanity.com/paper/2302.08764/) (2023)
- [Explicit and Implicit Knowledge Distillation via Unlabeled Data](https://mlsanity.com/paper/2302.08771/) (2023)
- [Towards Simultaneous Segmentation of Liver Tumors and Intrahepatic Vessels via Cross-attention Mechanism](https://mlsanity.com/paper/2302.09785/) (2023)
- [A novel efficient Multi-view traffic-related object detection framework](https://mlsanity.com/paper/2302.11810/) (2023)

## Built on by (most cited first, 30 of 34)

- [Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception](https://mlsanity.com/paper/2307.13929/) (2023)
- [The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification](https://mlsanity.com/paper/2305.17891/) (2023)
- [AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perception](https://mlsanity.com/paper/2307.13933/) (2023)
- [Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2404.16456/) (2024)
- [Towards Multimodal Sentiment Analysis Debiasing via Bias Purification](https://mlsanity.com/paper/2403.05023/) (2024)
- [Robust Emotion Recognition in Context Debiasing](https://mlsanity.com/paper/2403.05963/) (2024)
- [Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding](https://mlsanity.com/paper/2511.22850/) (2025)
- [VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoning](https://mlsanity.com/paper/2404.07078/) (2024)
- [Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning](https://mlsanity.com/paper/2411.02793/) (2024)
- [Improving Generalization in Visual Reinforcement Learning via Conflict-aware Gradient Agreement Augmentation](https://mlsanity.com/paper/2308.01194/) (2023)
- [Can LLMs' Tuning Methods Work in Medical Multimodal Domain?](https://mlsanity.com/paper/2403.06407/) (2024)
- [Large Vision-Language Models as Emotion Recognizers in Context Awareness](https://mlsanity.com/paper/2407.11300/) (2024)
- [Affective Image Editing: Shaping Emotional Factors via Text Descriptions](https://mlsanity.com/paper/2505.18699/) (2025)
- [De-confounded Data-free Knowledge Distillation for Handling Distribution Shifts](https://mlsanity.com/paper/2403.19539/) (2024)
- [Towards Context-Aware Emotion Recognition Debiasing from a Causal Demystification Perspective via De-confounded Training](https://mlsanity.com/paper/2407.04963/) (2024)
- [Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations](https://mlsanity.com/paper/2407.04955/) (2024)
- [Contextual Emotion Recognition using Large Vision Language Models](https://mlsanity.com/paper/2405.08992/) (2024)
- [Sampling to Distill: Knowledge Transfer from Open-World Data](https://mlsanity.com/paper/2307.16601/) (2023)
- [Human 3D Avatar Modeling with Implicit Neural Representation: A Brief Survey](https://mlsanity.com/paper/2306.03576/) (2023)
- [Visual and Textual Prompts in VLLMs for Enhancing Emotion Recognition](https://mlsanity.com/paper/2504.17224/) (2025)
- [Part-Whole Relational Fusion Towards Multi-Modal Scene Understanding](https://mlsanity.com/paper/2410.14944/) (2024)
- [Towards Deeper Emotional Reflection: Crafting Affective Image Filters with Generative Priors](https://mlsanity.com/paper/2512.17376/) (2025)
- [HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular Images](https://mlsanity.com/paper/2403.07912/) (2024)
- [Self-Cooperation Knowledge Distillation for Novel Class Discovery](https://mlsanity.com/paper/2407.01930/) (2024)
- [Fuzzy-aware Loss for Source-free Domain Adaptation in Visual Emotion Recognition](https://mlsanity.com/paper/2501.15519/) (2025)
- [Empowering Vision Transformers with Multi-Scale Causal Intervention for Long-Tailed Image Classification](https://mlsanity.com/paper/2505.08173/) (2025)
- [Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2508.04999/) (2025)
- [Multi-Scale Heterogeneity-Aware Hypergraph Representation for Histopathology Whole Slide Images](https://mlsanity.com/paper/2404.19334/) (2024)
- [TCAN: Text-oriented Cross Attention Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2404.04545/) (2024)
- [Causality-Driven Infrared and Visible Image Fusion](https://mlsanity.com/paper/2505.20830/) (2025)

## Abstract (arXiv)

Context-Aware Emotion Recognition (CAER) is a crucial and challenging task that aims to perceive the emotional states of the target person with contextual information. Recent approaches invariably focus on designing sophisticated architectures or mechanisms to extract seemingly meaningful representations from subjects and contexts. However, a long-overlooked issue is that a context bias in existing datasets leads to a significantly unbalanced distribution of emotional states among different context scenarios. Concretely, the harmful bias is a confounder that misleads existing models to learn spurious correlations based on conventional likelihood estimation, significantly limiting the models' performance. To tackle the issue, this paper provides a causality-based perspective to disentangle the models from the impact of such bias, and formulate the causalities among variables in the CAER task via a tailored causal graph. Then, we propose a Contextual Causal Intervention Module (CCIM) based on the backdoor adjustment to de-confound the confounder and exploit the true causal effect for model training. CCIM is plug-in and model-agnostic, which improves diverse state-of-the-art approaches by considerable margins. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our CCIM and the significance of causal insight.