mlsanityData to 14 Oct 2026

Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMs

arXiv 2508.19111 ↗First posted 26 Aug 2025cs.CLcited by 4

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Large vision-language models (LVLMs) demonstrate strong visual question answering (VQA) capabilities but are shown to hallucinate. A reliable model should perceive its knowledge boundaries-knowing what it knows and what it does not. This paper investigates LVLMs' perception of their knowledge boundaries by evaluating three types of confidence signals: probabilistic confidence, answer consistency-based confidence, and verbalized confidence. Experiments on three LVLMs across three VQA datasets show that, although LVLMs possess a reasonable perception level, there is substantial room for improvement. Among the three confidences, probabilistic and consistency-based signals are more reliable indicators, while verbalized confidence often leads to overconfidence. To enhance LVLMs' perception, we adapt several established confidence calibration methods from Large Language Models (LLMs) and propose three effective methods. Additionally, we compare LVLMs with their LLM counterparts, finding that jointly processing visual and textual inputs decreases question-answering performance but reduces confidence, resulting in an improved perception level compared to LLMs.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 36 references that are in the map. First posted on arXiv 26 Aug 2025.

Builds on

28 of 36 references.

References · 28

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  6. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  7. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  8. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  9. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  10. 2015Visual7W: Grounded Question Answering in Imagescited by 318
  11. 2020Calibration of Pre-trained Transformerscited by 315
  12. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
16 more
  1. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decodingcited by 203
  2. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  3. 2023Do Language Models Know When They're Hallucinating References?cited by 116
  4. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  5. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 90
  6. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedbackcited by 80
  7. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 73
  8. 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
  9. 2025Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Homecited by 33
  10. 2025Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perceptioncited by 27
  11. 2024Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence?cited by 23
  12. 2025Cross-Modal Attention Calibration for LVLM Hallucination Mitigationcited by 23
  13. 2024Just rephrase it! Uncertainty estimation in closed-source language models via multiple rephrased queriescited by 15
  14. 2024Perplexed: Understanding When Large Language Models are Confusedcited by 9
  15. 2025Detecting Knowledge Boundary of Vision Large Language Models by Sampling-Based Inferencecited by 6
  16. 2025Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularitycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Enhancing Trustworthy GUI Grounding via Self-Critiqued Reinforcement Learningcited by 3
  2. 2026How Do LLMs and VLMs Understand Viewpoint Rotation Without Vision? An Interpretability Studycited by 1
  3. 2026Prompt-Induced Score Variance in Zero-Shot Binary Vision-Language Safety Classificationcited by 1
  4. 2026OmniConfess: Eliciting Token Confessions to Mitigate Omni-Modal Hallucinationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024TUBench: Benchmarking Large Vision-Language Models on Trustworthiness with Unanswerable Questions
  2. 2024Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering
  3. 2024Unveiling the Tapestry of Consistency in Large Vision-Language Models
  4. 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs
  5. 2025Where do Large Vision-Language Models Look at when Answering Questions?
  6. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs
  7. 2024Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly
  8. 2026Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark