Omni-Perception Policy Optimization for Multimodal Emotion Reasoning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
We find that current emotion-oriented Omni-MLLMs still lack reliable omni-modal perception: they (i) underutilize multimodal cues in their reasoning trajectories and (ii) exhibit unfaithful behavior, often hallucinating modality-specific statements from other modalities. Building on these insights, we propose OPPO (Omni-Perception Policy Optimization), a reinforcement learning framework that explicitly optimizes multimodal perception. First, an Omni-Perception Reward decomposes ground-truth reasoning into fine-grained visual, acoustic, and emotion cues and rewards trajectories that semantically recover these cues. Second, an Omni-Perception Loss compares the policy under full and unimodally masked inputs, applying a KL penalty only to modality-specific evidence tokens to suppress cross-modal hallucination. We further introduce MEP-Bench, a diagnostic benchmark that quantifies utilization and faithfulness. Experiments show that OPPO achieves state-of-the-art performance on MER-UniBench and MME-Emotion, while substantially improving utilization and faithfulness scores on MEP-Bench, highlighting the importance of sufficient and faithful omni perception for multimodal emotion reasoning.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 51 references that are in the map. First posted on arXiv 24 Jun 2026.
Builds on
14 of 51 references.
Closest ancestors · 2
Same direction, newest first
- 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Context26 Jun
- 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward8 Jun
Other references · 12
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024GPT-4o System Cardcited by 4,588
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
- 2025OpenAI GPT-5 System Cardcited by 989
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
- 2025Qwen3-Omni Technical Reportcited by 497
- 2025R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learningcited by 100
- 2025Psyche-R1: Towards Reliable Psychological LLMs through Unified Empathy, Expertise, and Reasoningcited by 20
- 2025AffectGPT-R1: Leveraging Reinforcement Learning for Open-Vocabulary Multimodal Emotion Recognitioncited by 15
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judgingcited by 1
- 2026ProFocus: Interpreting Affective Experience in Artistic Images with Progressive Visual Focusingcited by 0
- 2026Affect-Prototype Guided Fusion for Open-Vocabulary Incomplete Multi-modal Emotion Recognitioncited by 0
- 2026Affect-Prototype Guided Fusion for Open-Vocabulary Incomplete Multi-modal Emotion Recognition15 Sep 2026
- 2026ProFocus: Interpreting Affective Experience in Artistic Images with Progressive Visual Focusing14 Aug 2026
- 2026Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging11 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing
- 2025Omni-AutoThink: Adaptive Multimodal Reasoning via Reinforcement Learning
- 2025OmniDPO: A Preference Optimization Framework to Address Omni-Modal Hallucination
- 2026CFPO: Counterfactual Policy Optimization for Multimodal Reasoning
- 2026OneEmo: A Unified Multimodal Reasoning Model for Emotion Perception, Understanding, and Interaction
- 2025Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
- 2025Perception-Aware Policy Optimization for Multimodal Reasoning
- 2026MER-R1: Multimodal Emotion Reasoning via Slow-Fast Thinking Synergy