mlsanityData to 14 Oct 2026

OViP: Online Vision-Language Preference Learning for VLM Hallucination

arXiv 2505.15963 ↗First posted 21 May 2025cs.CV cs.CLcited by 2

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

also in 1 more
Abstract (arXiv)

Large vision-language models (LVLMs) remain vulnerable to hallucination, often generating content misaligned with visual inputs. Although recent training-based approaches aim to mitigate hallucination, they typically rely on predefined or randomly edited negative samples that do not reflect actual model errors, thus limiting training efficacy. In this work, we propose an Online Vision-language Preference Learning (OViP) framework that dynamically constructs contrastive training data based on the model's own hallucinated outputs. By identifying semantic differences between sampled response pairs and synthesizing negative images using a diffusion model, OViP generates more relevant supervision signals in real time. This failure-driven training enables adaptive alignment of both textual and visual preferences. Moreover, we refine existing evaluation protocols to better capture the trade-off between hallucination suppression and expressiveness. Experiments on hallucination and general benchmarks demonstrate that OViP not only reduces hallucinations while preserving core multi-modal capabilities, but also substantially improves training efficiency. Code is available at https://github.com/lsjlsj35/Online-Vision-Language-Preference-Learning-for-VLM-Hallucination.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 52 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

46 of 52 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs28 Jan
  2. 2025Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key16 Jan
  3. 2024V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization5 Nov
  4. 2024CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs19 Aug
  5. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  6. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback22 Apr
  7. 2024ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models24 Mar
  8. 2024A Survey on Hallucination in Large Vision-Language Models1 Feb
  9. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  10. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  11. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  12. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
2 more
  1. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  2. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 32

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2023Qwen Technical Reportcited by 3,344
  10. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  12. 2019Towards VQA Models That Can Readcited by 1,718
20 more
  1. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  3. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  4. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  5. 2018Object Hallucination in Image Captioningcited by 669
  6. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  7. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  8. 2025Learning to Reason under Off-Policy Guidancecited by 281
  9. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 265
  10. 2024Direct Language Model Alignment from Online AI Feedbackcited by 260
  11. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
  12. 2024mDPO: Conditional Preference Optimization for Multimodal Large Language Modelscited by 92
  13. 2024Calibrated Self-Rewarding Vision Language Modelscited by 78
  14. 2024VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Modelscited by 43
  15. 2024ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modelingcited by 34
  16. 2024Understanding Alignment in Multimodal LLMs: A Comprehensive Studycited by 23
  17. 2024Synth2: Boosting Visual-Language Models with Synthetic Captions and Image Embeddingscited by 18
  18. 2024Active Learning for Vision-Language Modelscited by 15
  19. 2025Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Imagescited by 14
  20. 2025AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimizationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026Visual Preference Optimization with Rubric Rewardscited by 5
  2. 2025Behavioral Geometric Supervision Aligns Video Foundation Models with Human Social Perceptioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Online Self-Calibration Against Hallucination in Vision-Language Models
  2. 2026Grounded Post-Training with Hard Examples for Reducing Hallucination in Multimodal Large Language Models
  3. 2025Stop learning it all to mitigate visual hallucination, Focus on the hallucination target
  4. 2026Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs
  5. 2025Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting Hallucinations
  6. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
  7. 2025Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization
  8. 2025Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization