mlsanityData to 14 Oct 2026

Multimodal Video Emotion Recognition with Reliable Reasoning Priors

arXiv 2508.03722 ↗First posted 29 Jul 2025cs.CV cs.AIcited by 3

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

This study investigates the integration of trustworthy prior reasoning knowledge from MLLMs into multimodal emotion recognition. We employ Gemini to generate fine-grained, modality-separable reasoning traces, which are injected as priors during the fusion stage to enrich cross-modal interactions. To mitigate the pronounced class-imbalance in multimodal emotion recognition, we introduce Balanced Dual-Contrastive Learning, a loss formulation that jointly balances inter-class and intra-class distributions. Applied to the MER2024 benchmark, our prior-enhanced framework yields substantial performance gains, demonstrating that the reliability of MLLM-derived reasoning can be synergistically combined with the domain adaptability of lightweight fusion networks for robust, scalable emotion recognition.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 35 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

21 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning17 Jun

Other references · 20

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  5. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  6. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  7. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  8. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  9. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  10. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  11. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  12. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
8 more
  1. 2022Balanced Contrastive Learning for Long-Tailed Visual Recognitioncited by 144
  2. 2023MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learningcited by 33
  3. 2024MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognitioncited by 23
  4. 2024OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognitioncited by 17
  5. 2024Multimodal Emotion Recognition with Vision-language Prompting and Modality Dropoutcited by 7
  6. 2024Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross Attentioncited by 5
  7. 2025TACFN: Transformer-based Adaptive Cross-modal Fusion Network for Multimodal Emotion Recognitioncited by 3
  8. 2024LaMI: Augmenting Large Language Models via Late Multi-Image Fusioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documentscited by 2
  2. 2026Position: Life-Logging Video Streams Make the Privacy-Utility Trade-off Inevitablecited by 1
  3. 2026Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Emotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier
  2. 2025Beyond Emotion Recognition: A Multi-Turn Multimodal Emotion Understanding and Reasoning Benchmark
  3. 2025Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey
  4. 2025Pioneering Multimodal Emotion Recognition in the Era of Large Models: From Closed Sets to Open Vocabularies
  5. 2026Follow the Clues, Frame the Truth: Hybrid-evidential Deductive Reasoning in Open-Vocabulary Multimodal Emotion Recognition
  6. 2026SURE: Synergistic Uncertainty-aware Reasoning for Multimodal Emotion Recognition in Conversations
  7. 2026Rationale-Guided Learning for Multimodal Emotion Recognition
  8. 2026Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding