mlsanityData to 7 Oct 2026

Follow the Clues, Frame the Truth: Hybrid-evidential Deductive Reasoning in Open-Vocabulary Multimodal Emotion Recognition

arXiv 2603.16463 ↗First posted 17 Mar 2026cs.AI cs.HCcited by 0

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Open-Vocabulary Multimodal Emotion Recognition (OV-MER) is inherently challenging due to the ambiguity of equivocal multimodal cues, which often stem from distinct unobserved situational dynamics. While Multimodal Large Language Models (MLLMs) offer extensive semantic coverage, their performance is often bottlenecked by premature commitment to dominant data priors, resulting in suboptimal heuristics that overlook crucial, complementary affective cues across modalities. We argue that effective affective reasoning requires more than surface-level association; it necessitates reconstructing nuanced emotional states by synthesizing multiple evidence-grounded rationales that reconcile these observations from diverse latent perspectives. We introduce HyDRA, a Hybrid-evidential Deductive Reasoning Architecture that formalizes inference as a Propose-Verify-Decide protocol. To internalize this abductive process, we employ reinforcement learning with hierarchical reward shaping, aligning the reasoning trajectories with final task performance to ensure they best reconcile the observed multimodal cues. Systematic evaluations validate our design choices, with HyDRA consistently outperforming strong baselines--especially in ambiguous or conflicting scenarios--while providing interpretable, diagnostic evidence traces.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 61 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

19 of 61 references.

References · 19

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  3. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  4. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  5. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  6. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
  7. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  8. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  9. 2025R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learningcited by 99
  10. 2025HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understandingcited by 58
  11. 2025Grounded Chain-of-Thought for Multimodal Large Language Modelscited by 57
  12. 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 42
7 more
  1. 2025VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understandingcited by 32
  2. 2025Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibrationcited by 23
  3. 2023Explainable Multimodal Emotion Recognitioncited by 18
  4. 2024OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognitioncited by 17
  5. 2025On the Fundamental Limits of LLMs at Scalecited by 16
  6. 2024Large Vision-Language Models as Emotion Recognizers in Context Awarenesscited by 12
  7. 2024Video Emotion Open-vocabulary Recognition Based on Multimodal Large Language Modelcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning What to Attend First: Modality-Importance-Guided Reasoning for Reliable Multimodal Emotion Understanding
  2. 2025Emotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier
  3. 2025Pioneering Multimodal Emotion Recognition in the Era of Large Models: From Closed Sets to Open Vocabularies
  4. 2026MER-R1: Multimodal Emotion Reasoning via Slow-Fast Thinking Synergy
  5. 2026Rationale-Guided Learning for Multimodal Emotion Recognition
  6. 2025AffectGPT-R1: Leveraging Reinforcement Learning for Open-Vocabulary Multimodal Emotion Recognition
  7. 2025Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey
  8. 2026SURE: Synergistic Uncertainty-aware Reasoning for Multimodal Emotion Recognition in Conversations