mlsanityData to 14 Oct 2026

DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs

arXiv 2609.28570 ↗First posted 23 Sep 2026cs.AI cs.LGcited by 0

On the map: LLM reasoning → Hallucination detection and mitigation

also in 1 more
Abstract (arXiv)

Reinforcement learning (RL) is widely used to sharpen reasoning in multimodal large language models (MLLMs), yet its effect on hallucination is uneven. We trace this to two weak points in the correction chain from reward to parameter update. At the rollout level, hard queries---those with high semantic entropy---frequently produce unanimously wrong sample groups, collapsing the group-relative advantage to zero exactly where hallucination risk is highest. At the optimization level, confident-but-wrong tokens are gradient-invisible: a categorical policy's expected score-gradient norm vanishes as its distribution sharpens, so the predictions that most need correction receive the weakest updates. We propose Dual-Entropy Enhanced Policy Optimization (DEEPO), a dual-stage enhancement combining signal variance regularization with gradient preconditioning: semantic-entropy-triggered expert prefixes inject grounded continuations on high-uncertainty queries, providing direct supervision and restoring advantage variance, while advantage-sign-aware Renyi preconditioning counteracts logit-level saturation so correction reaches confident errors in the operational confidence regime. Both branches improve over GRPO individually; their interaction is statistically significant on VideoMMMU---the most complex long-horizon task in our evaluation suite (+4.0, 95% CI [1.1, 6.9])---and additive elsewhere. DEEPO reduces hallucination while preserving accuracy and training stability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 49 of its 65 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

49 of 65 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibration4 Feb
  2. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning30 Aug

Other references · 47

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  7. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  8. 2023A Survey on Multimodal Large Language Modelscited by 909
  9. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  11. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  12. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 484
35 more
  1. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  2. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  3. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 316
  4. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
  5. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 225
  6. 2025Why Language Models Hallucinatecited by 220
  7. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
  8. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 162
  9. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
  10. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 99
  11. 2025HalluLens: LLM Hallucination Benchmarkcited by 92
  12. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 91
  13. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoningcited by 84
  14. 2025Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Keycited by 65
  15. 2024Multi-Object Hallucination in Vision-Language Modelscited by 60
  16. 2025Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewardscited by 50
  17. 2023Mitigating Hallucination in Visual Language Models with Visual Supervisioncited by 49
  18. 2024Calibrating the Confidence of Large Language Models by Eliciting Fidelitycited by 47
  19. 2024FLAME: Factuality-Aware Alignment for Large Language Modelscited by 47
  20. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidancecited by 46
  21. 2025Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agentscited by 46
  22. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  23. 2024Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answeringcited by 32
  24. 2024Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimizationcited by 20
  25. 2024Enhancing Advanced Visual Reasoning Ability of Large Language Modelscited by 18
  26. 2025The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hackingcited by 18
  27. 2025A comprehensive taxonomy of hallucinations in Large Language Modelscited by 18
  28. 2025SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesiscited by 12
  29. 2025Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Modelscited by 7
  30. 2024Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMscited by 6
  31. 2026Mitigating Manifold Departure: Uncertainty-Aware Subspace Rectification for Trustworthy MLLM Decodingcited by 4
  32. 2026Reconciling Process Supervision with Outcome-Based Credit in Agentic Policy Optimizationcited by 4
  33. 2026SARE: Sample-wise Adaptive Reasoning for Training-free Fine-grained Visual Recognitioncited by 3
  34. 2025The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMscited by 2
  35. 2026Dual-Axis Policy Optimization for LLM Agents: Bayesian Feedback Attribution and Trajectory Mass Normalizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation
  2. 2025Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models
  3. 2025Optimizing LVLMs with On-Policy Data for Effective Hallucination Mitigation
  4. 2025LLM-CAS: Dynamic Neuron Perturbation for Real-Time Hallucination Correction
  5. 2026Ground What You See: Hallucination-Resistant MLLMs via Caption Feedback, Diversity-Aware Sampling, and Conflict Regularization
  6. 2026IRIS: Implicit Reward-Guided Internal Sifting for Mitigating Multimodal Hallucination
  7. 2025Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinations
  8. 2025COPO: Causal-Oriented Policy Optimization for Hallucinations of MLLMs