mlsanityData to 7 Oct 2026

RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

arXiv 2312.00849 ↗First posted 1 Dec 2023cs.CL cs.CVcited by 404

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have recently demonstrated impressive capabilities in multimodal understanding, reasoning, and interaction. However, existing MLLMs prevalently suffer from serious hallucination problems, generating text that is not factually grounded in associated images. The problem makes existing MLLMs untrustworthy and thus impractical in real-world (especially high-stakes) applications. To address the challenge, we present RLHF-V, which enhances MLLM trustworthiness via behavior alignment from fine-grained correctional human feedback. Specifically, RLHF-V collects human preference in the form of segment-level corrections on hallucinations, and performs dense direct preference optimization over the human feedback. Comprehensive experiments on five benchmarks in both automatic and human evaluation show that, RLHF-V can enable substantially more trustworthy MLLM behaviors with promising data and computation efficiency. Remarkably, using 1.4k annotated data samples, RLHF-V significantly reduces the hallucination rate of the base MLLM by 34.8%, outperforming the concurrent LLaVA-RLHF trained on 10k annotated data. The final model achieves state-of-the-art performance in trustworthiness among open-source MLLMs, and shows better robustness than GPT-4V in preventing hallucinations aroused from over-generalization. We open-source our code, model, and data at https://github.com/RLHF-V/RLHF-V.

Key facts. Cited by 404 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 49 of its 64 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

49 of 64 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  2. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  3. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
34 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2023Let's Verify Step by Stepcited by 3,598
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  9. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  11. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  12. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  13. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  14. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  15. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  16. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  17. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  18. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  19. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  20. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  21. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  22. 2018Object Hallucination in Image Captioningcited by 661
  23. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  24. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  25. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  26. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  27. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  28. 2021Alignment of Language Agentscited by 167
  29. 2023M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuningcited by 126
  30. 2023VIGC: Visual Instruction Generation and Correctioncited by 80
  31. 2023On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Drivingcited by 79
  32. 2023Reformulating Vision-Language Foundation Models and Datasets Towards Universal Multimodal Assistantscited by 25
  33. 2023Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characterscited by 21
  34. 2023SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understandingcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 125; 2025: 162; 2026: 116.

404 papers.

Sort:

By area:

  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  2. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  3. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  4. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  5. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  6. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  7. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  8. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 238
  9. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  10. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  11. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  12. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 193
38 more
  1. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 172
  2. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 171
  3. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  4. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  5. 2024LLaVA-Critic: Learning to Evaluate Multimodal Modelscited by 132
  6. 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 129
  7. 2023Silkie: Preference Distillation for Large Visual Language Modelscited by 128
  8. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  9. 2024GUICourse: From General Vision Language Models to Versatile GUI Agentscited by 112
  10. 2024Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Modelscited by 97
  11. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  12. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 96
  13. 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
  14. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Modelscited by 90
  15. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
  16. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  17. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 86
  18. 2024VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Modelscited by 85
  19. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  20. 2024What If We Recaption Billions of Web Images with LLaMA-3?cited by 82
  21. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedbackcited by 80
  22. 2024Calibrated Self-Rewarding Vision Language Modelscited by 77
  23. 2024Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvementcited by 74
  24. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Modelcited by 73
  25. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Modelscited by 72
  26. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  27. 2024Vript: A Video Is Worth Thousands of Wordscited by 71
  28. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learningcited by 68
  29. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  30. 2024VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understandingcited by 65
  31. 2025Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Keycited by 64
  32. 2024MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMscited by 62
  33. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  34. 2024Protecting Privacy in Multimodal Large Language Models with MLLMU-Benchcited by 61
  35. 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuningcited by 61
  36. 2024Multi-Object Hallucination in Vision-Language Modelscited by 60
  37. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 55
  38. 2025Visual Agentic Reinforcement Fine-Tuningcited by 53

50 most cited of 404 All 404 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization
  2. 2025Taming Object Hallucinations with Verified Atomic Confidence Estimation
  3. 2026Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs
  4. 2025INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling
  5. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization
  6. 2026Grounded Post-Training with Hard Examples for Reducing Hallucination in Multimodal Large Language Models
  7. 2025Confidence-Aware Routing for Large Language Model Reliability Enhancement: A Multi-Signal Approach to Pre-Generation Hallucination Mitigation
  8. 2026KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluation