mlsanityData to 14 Oct 2026

Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning

arXiv 2609.36782 ↗First posted 29 Sep 2026cs.CVcited by 0

On the map: LLM reasoning → Audio-language model reasoning

also in 1 more
Abstract (arXiv)

While multimodal large language models (MLLMs) have demonstrated exceptional capabilities in objective understanding tasks, their performance in affective reasoning still falls significantly short of human standards. We attribute it to a central capability gap: MLLMs are difficult to reliably distinguish semantically proximal emotions based on fine-grained visual evidence, which could be decoupled as two limitations: 1) Insufficient Attribution. The global reasoning paradigm of conventional MLLMs severely dilutes fine-grained emotion cues, where subtle emotional states are usually implicitly encoded, thereby generating emotional misjudgments in complex scenarios. 2) Insufficient Discrimination. Existing methods could only identify regions generally associated with emotions, which fails to distinguish discriminative regions between semantically similar emotions, leading to ambiguous emotion judgements. To overcome these limitations, we present a training-free inference-time optimization framework, named Decoding Affective Nuances (DAN). Specifically, we propose a Hierarchical Emotional Reasoning Chain (HERC) that enhances the insufficient attribution by harmonizing fine-grained scene/object-level cues and performing a soft-gated reasoning. Furthermore, to discriminate between semantically proximal emotions, we design a Contrastive Discriminative Visual Pruning (CDVP), which isolates discriminative visual tokens to reason the final emotion category by computing the absolute discrepancy between the attention distributions of similar emotions. Performances on several benchmarks demonstrate that DAN significantly improves discrimination for affective nuances without consuming additional training resources, especially achieving +10.47% improvements with Qwen3-VL-8B-Instruct on WebEmo25 dataset that contains 25 fine-grained emotion categories.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 46 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

31 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models27 Feb
  2. 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning17 Jun

Other references · 29

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  6. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  7. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  8. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 164
  9. 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 161
  10. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 153
  11. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Modelscited by 113
  12. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 93
17 more
  1. 2023EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributescited by 82
  2. 2016Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmarkcited by 71
  3. 2025AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Modelscited by 44
  4. 2018Contemplating Visual Emotions: Understanding and Overcoming Dataset Biascited by 41
  5. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 32
  6. 2022Multi-Attention Network for Compressed Video Referring Object Segmentationcited by 28
  7. 2024Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detectioncited by 20
  8. 2025Towards Efficient Partially Relevant Video Retrieval with Active Moment Discoveringcited by 18
  9. 2024Dual-path Collaborative Generation Network for Emotional Video Captioningcited by 14
  10. 2023Improving Image Captioning via Predicting Structured Conceptscited by 13
  11. 2024Visual Prompting in LLMs for Enhancing Emotion Recognitioncited by 12
  12. 2025MSE-Adapter: A Lightweight Plugin Endowing LLMs with the Capability to Perform Multimodal Sentiment Analysis and Emotion Recognitioncited by 9
  13. 2026CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layerscited by 9
  14. 2025Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learningcited by 7
  15. 2025Zero-Shot Fine-Grained Image Classification Using Large Vision-Language Modelscited by 4
  16. 2026EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generationcited by 3
  17. 2025EmoGist: Efficient In-Context Learning for Visual Emotion Understandingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models
  2. 2026From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding
  3. 2025Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach
  4. 2025Learning What to Attend First: Modality-Importance-Guided Reasoning for Reliable Multimodal Emotion Understanding
  5. 2026Integrating Fine-Grained Audio-Visual Evidence for Robust Multimodal Emotion Reasoning
  6. 2024EmoLLM: Multimodal Emotional Understanding Meets Large Language Models
  7. 2025Emotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier
  8. 2025EmoCaliber: Advancing Reliable Visual Emotion Comprehension via Confidence Verbalization and Calibration