# Learning What to Attend First: Modality-Importance-Guided Reasoning for Reliable Multimodal Emotion Understanding

arXiv: https://arxiv.org/abs/2512.02699 · HTML: https://mlsanity.com/paper/2512.02699/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 2 Dec 2025.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)
- Multimodal LLMs / vision-language models / [Multimodal emotion understanding](https://mlsanity.com/mllm/multimodal-emotion-understanding/)

## Builds on (most cited first, 22 of 22)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model](https://mlsanity.com/paper/2504.07615/) (2025)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2504.06958/) (2025)
- [R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning](https://mlsanity.com/paper/2503.05379/) (2025)
- [HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context](https://mlsanity.com/paper/2506.21277/) (2025)
- [GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning](https://mlsanity.com/paper/2506.16141/) (2025)
- [FaceXFormer: A Unified Transformer for Facial Analysis](https://mlsanity.com/paper/2403.12960/) (2024)
- [EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learning](https://mlsanity.com/paper/2505.04623/) (2025)
- [EMO-LLaMA: Enhancing Facial Emotion Understanding with Instruction Tuning](https://mlsanity.com/paper/2408.11424/) (2024)
- [Omni-Emotion: Extending Video MLLM with Detailed Face and Audio Modeling for Multimodal Emotion Analysis](https://mlsanity.com/paper/2501.09502/) (2025)
- [Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey](https://mlsanity.com/paper/2509.24322/) (2025)
- [ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding](https://mlsanity.com/paper/2506.01274/) (2025)
- [When Thinking Drifts: Evidential Grounding for Robust Video Reasoning](https://mlsanity.com/paper/2510.06077/) (2025)
- [GTR: Guided Thought Reinforcement Prevents Thought Collapse in RL-based VLM Agent Training](https://mlsanity.com/paper/2503.08525/) (2025)
- [DIP-R1: Deep Inspection and Perception with RL Looking Through and Understanding Complex Scenes](https://mlsanity.com/paper/2505.23179/) (2025)
- [FaVChat: Hierarchical Prompt-Query Guided Facial Video Understanding with Data-Efficient GRPO](https://mlsanity.com/paper/2503.09158/) (2025)
- [FEALLM: Advancing Facial Emotion Analysis in Multimodal Large Language Models with Emotional Synergy and Reasoning](https://mlsanity.com/paper/2505.13419/) (2025)
- [Emotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier](https://mlsanity.com/paper/2510.23506/) (2025)

## Built on by (most cited first, 1 of 1)

- [MAPLE: Modality-Aware Post-training and Learning Ecosystem](https://mlsanity.com/paper/2602.11596/) (2026)

## Abstract (arXiv)

In this paper, we present Modality-Importance-Guided Reasoning (MIGR), a framework designed to improve the reliability of reasoning-based multimodal emotion understanding in multimodal large language models. Although existing methods have advanced emotion understanding, they often suffer from reasoning drift: models gradually rely on their own generated text instead of multimodal evidence, and their explanations are overly shaped by visually initiated reasoning paths. To address these issues, we introduce Modality Importance (MI), a simple yet effective mechanism for identifying the emotion-dominant modality. Using MI, MIGR reorganizes reasoning sequences so that explanations begin from the modality most critical to the target emotion, preventing early reasoning from being misled by less informative cues. Our two-stage framework-comprising modality-aligned supervised fine-tuning and modality-aware reward optimization-encourages models to generate emotionally grounded, causally relevant, and coherence-preserving explanations. Experimental results on the DFEW benchmark show that MIGR substantially improves reasoning reliability, decreasing instances of correct predictions accompanied by emotionally inconsistent explanations from 18.10% to 7.37%. These results confirm the benefit of initiating reasoning from the emotion-dominant modality.