mlsanityData to 7 Oct 2026

Delineating Knowledge Boundaries for Honest Large Vision-Language Models

arXiv 2604.26419 ↗First posted 29 Apr 2026cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Large Vision-Language Models (VLMs) have achieved remarkable multimodal performance yet remain prone to factual hallucinations, particularly in long-tail or specialized domains. Moreover, current models exhibit a weak capacity to refuse queries that exceed their parametric knowledge. In this paper, we propose a systematic framework to enhance the refusal capability of VLMs when facing such unknown questions. We first curate a model-specific "Visual-Idk" (Visual-I don't know) dataset, leveraging multi-sample consistency probing to distinguish between known and unknown facts. We then align the model using supervised fine-tuning followed by preference-aware optimization (e.g., DPO, ORPO) to effectively delineate its knowledge boundaries. Results on the Visual-Idk dataset show our method improves the Truthful Rate from 57.9% to 67.3%. Additionally, internal probing also demonstrates that the model genuinely recognizes its boundaries instead of just memorizing refusal patterns. Our framework further generalizes to out-of-distribution medical and perceptual domains, providing a robust path toward more trustworthy and prudent visual assistants.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 26 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

24 of 26 references.

References · 24

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  9. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  10. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  11. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  12. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
12 more
  1. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  2. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
  3. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  4. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 223
  5. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  6. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 106
  7. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  8. 2024V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimizationcited by 67
  9. 2023The Wisdom of Hindsight Makes Language Models Better Instruction Followerscited by 59
  10. 2024CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMscited by 49
  11. 2024Can AI Assistants Know What They Don't Know?cited by 47
  12. 2024Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusalcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Uncertainty-Aware Decision Making in Multimodal Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diagnosing Visual Ignorance in Vision-Language Models
  2. 2024Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models
  3. 2025MoHoBench: Assessing Honesty of Multimodal Large Language Models via Unanswerable Visual Questions
  4. 2024TUBench: Benchmarking Large Vision-Language Models on Trustworthiness with Unanswerable Questions
  5. 2026CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models
  6. 2025SegSub: Evaluating Robustness to Knowledge Conflicts and Hallucinations in Vision-Language Models
  7. 2026VirtueBench: Evaluating Trustworthiness under Uncertainty in Long Video Understanding
  8. 2024Unveiling the Tapestry of Consistency in Large Vision-Language Models