mlsanityData to 7 Oct 2026

VORD: Visual Ordinal Calibration for Mitigating Object Hallucinations in Large Vision-Language Models

arXiv 2412.15739 ↗First posted 20 Dec 2024cs.CVcited by 4

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large Vision-Language Models (LVLMs) have made remarkable developments along with the recent surge of large language models. Despite their advancements, LVLMs have a tendency to generate plausible yet inaccurate or inconsistent information based on the provided source content. This phenomenon, also known as ``hallucinations" can have serious downstream implications during the deployment of LVLMs. To address this, we present VORD a simple and effective method that alleviates hallucinations by calibrating token predictions based on ordinal relationships between modified image pairs. VORD is presented in two forms: 1.) a minimalist training-free variant which eliminates implausible tokens from modified image pairs, and 2.) a trainable objective function that penalizes unlikely tokens. Our experiments demonstrate that VORD delivers better calibration and effectively mitigates object hallucinations on a wide-range of LVLM benchmarks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 33 of its 77 references that are in the map. First posted on arXiv 20 Dec 2024.

Builds on

33 of 77 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention18 Jun
  2. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  3. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  4. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct

Other references · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  10. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  12. 2019The Curious Case of Neural Text Degenerationcited by 2,881
17 more
  1. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  3. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  4. 2019When Does Label Smoothing Help?cited by 985
  5. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  6. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  7. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  8. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  9. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  10. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  11. 2023ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Modelscited by 116
  12. 2020Confidence-Aware Learning for Deep Neural Networkscited by 113
  13. 2024A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)cited by 44
  14. 2023What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifierscited by 35
  15. 2024An Empirical Study Into What Matters for Calibrating Vision-Language Modelscited by 20
  16. 2023RankMixup: Ranking-Based Mixup Training for Network Calibrationcited by 19
  17. 2024SQ-LLaVA: Self-Questioning for Large Vision-Language Assistantcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Hallucination Begins Where Saliency Dropscited by 8
  2. 2026When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMscited by 1
  3. 2026Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigationcited by 1
  4. 2026Evidence-Order Calibration for Selective Visual Reasoning under Progressive Loss of Question-Critical Evidencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
  2. 2025Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibration
  3. 2025A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models
  4. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  5. 2026NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors
  6. 2024H-POPE: Hierarchical Polling-based Probing Evaluation of Hallucinations in Large Vision-Language Models
  7. 2023Evaluating Object Hallucination in Large Vision-Language Models
  8. 2024A Survey on Hallucination in Large Vision-Language Models