Causal-EVC: Breaking Emotional Spurious Causality via Spatiotemporal Grounding and Counterfactual Intervention
Not on the map. Why?
Abstract (arXiv)
Emotional Video Captioning aims to generate factually accurate and emotionally empathetic descriptions. While recent methods have recognized the importance of visual causes to guide emotion perception and caption generation, they fundamentally rely on simple attention matching, which inevitably suffers from {causal redundancy and spurious correlations} in co-occurrence bias (e.g., misclassifying ``sadness'' as ``joy'' on a sunny beach), leading to severe shortcut learning from confusing backgrounds. Furthermore, existing evaluations fail to verify whether models have genuinely mastered causal reasoning or merely exploited background confounders. To address these limitations, we first construct {EVC-CauseGround}, a comprehensive benchmark with dense spatio-temporal causal annotations. Crucially, it introduces a carefully selected {Causal-Faithfulness Subset} to explicitly quantify genuine emotion-cause attribution. Second, we propose {Causal-EVC}, an emotion-grounding captioning framework, which introduces a Motion-guided Causal Spatiotemporal Localization module to precisely decouple causal triggers from background confounders. Besides, we introduce an Interpretable Sparse Emotion Routing module. By synthesizing counterfactual representations and formulating a novel counterfactual contrastive objective, we enforce the model to anchor its emotion predictions strictly on authentic causal triggers instead of confusing background. Extensive experiments show that Causal-EVC not only achieves the best performance on semantic metrics but also exhibits significant advantages in the causal-faithfulness subset, which demonstrates that our model could mine emotional cues from genuine visual causes and mitigate co-occurrence bias for interpretable multimodal emotion understanding.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 69 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
32 of 69 references.
References · 32
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2024GPT-4o System Cardcited by 4,467
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
- 2025OpenAI GPT-5 System Cardcited by 910
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
- 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
- 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
20 more
- 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
- 2020Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentencescited by 116
- 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 99
- 2018Reconstruction Network for Video Captioningcited by 98
- 2019Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Textscited by 82
- 2022MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wildcited by 65
- 2021Semantic Grouping Network for Video Captioningcited by 33
- 2021Multimodal Emotion-Cause Pair Extraction in Conversationscited by 30
- 2023Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Propertiescited by 21
- 2024Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detectioncited by 19
- 2025Continual Multimodal Contrastive Learningcited by 19
- 2024CaReBench: A Fine-Grained Benchmark for Video Captioning and Retrievalcited by 18
- 2025Towards Efficient Partially Relevant Video Retrieval with Active Moment Discoveringcited by 17
- 2025MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Modelscited by 17
- 2024Dual-path Collaborative Generation Network for Emotional Video Captioningcited by 13
- 2025HUME: Measuring the Human-Model Performance Gap in Text Embedding Taskscited by 7
- 2025IF-VidCap: Can Video Caption Models Follow Instructions?cited by 7
- 2023Emotion Prediction Oriented method with Multiple Supervisions for Emotion-Cause Pair Extractioncited by 4
- 2025CSPCL: Category Semantic Prior Contrastive Learning for Deformable DETR-Based Prohibited Item Detectorscited by 3
- 2026Adaptive Emotional Video Captioning via Affective Heterogeneous Graph Reasoning and Multi-task Joint Learningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Benchmarking Dynamic Affective Reasoning: A Viewer-Centric Video Emotion Dataset
- 2024StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models
- 2025Decoding the Flow: CauseMotion for Emotional Causality Analysis in Long-form Conversations
- 2025Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models
- 2026EmoWorld: A Decoupled Affective Field for Controllable Emotional Video Generation
- 2025EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysis
- 2026Video2Reaction: Training Foundation Video Models to Predict Audience Reaction
- 2026YoCausal: How Far is Video Generation from World Model? A Causality Perspective