mlsanityData to 7 Oct 2026

Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs

arXiv 2604.24395 ↗First posted 27 Apr 2026cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

also in 1 more
Abstract (arXiv)

Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary models to construct preference datasets. We identify that this reliance introduces a distributional mismatch between the proprietary and target models that hinders efficient alignment. To address this, we propose Alignment via VErified Self-correction DPO (AVES-DPO), a framework that aligns LVLMs using in-distribution data derived from the model's intrinsic knowledge. Our approach employs a consensus-based verification mechanism to diagnose diverse hallucinations and guides the model to self-correct, thereby generating preference pairs strictly compatible with its internal distribution. Extensive experiments demonstrate that AVES-DPO surpasses existing baselines in hallucination mitigation while requiring only 5.2k samples.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 34 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

23 of 34 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization27 Aug
  2. 2025Mitigating Object Hallucinations via Sentence-Level Early Intervention16 Jul
  3. 2025Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key16 Jan
  4. 2024CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs19 Aug
  5. 2024Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification2 Jul
  6. 2024Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment28 May
  7. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites4 Dec
  8. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  9. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  10. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  11. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision13 Nov
  12. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun

Other references · 11

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  7. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  8. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
  9. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  10. 2024Understanding Alignment in Multimodal LLMs: A Comprehensive Studycited by 22
  11. 2025DiscoSG: Towards Discourse-Level Text Scene Graph Parsing through Iterative Graph Refinementcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Hallucination Mitigation for Large Vision-Language Models via Implicit Feature Stabilizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting Hallucinations
  2. 2025TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction in MLLMs
  3. 2026Online Self-Calibration Against Hallucination in Vision-Language Models
  4. 2025Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization
  5. 2025Mitigating Visual Hallucinations via Semantic Curriculum Preference Optimization in MLLMs
  6. 2025OViP: Online Vision-Language Preference Learning for VLM Hallucination
  7. 2025Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization
  8. 2025Stop learning it all to mitigate visual hallucination, Focus on the hallucination target