mlsanityData to 7 Oct 2026

Context-Aware Decoding for Faithful Vision-Language Generation

arXiv 2601.05939 ↗First posted 9 Jan 2026cs.CVcited by 1

On the map: LLM reasoning → Hallucination detection and mitigation

also in 1 more
Abstract (arXiv)

Hallucinations, generating responses inconsistent with the visual input, remain a critical limitation of large vision-language models (LVLMs), especially in open-ended tasks such as image captioning and visual reasoning. In this work, we probe the layer-wise generation dynamics that drive hallucinations and propose a training-free mitigation strategy. Employing the Logit Lens, we examine how LVLMs construct next-token distributions across decoder layers, uncovering a pronounced commitment-depth gap: truthful tokens accumulate probability mass on their final candidates earlier than hallucinatory ones. Drawing on this discovery, we introduce Context Embedding Injection (CEI), a lightweight method that harnesses the hidden state of the last input token-the context embedding-as a grounding signal to maintain visual fidelity throughout decoding and curb hallucinations. Evaluated on the CHAIR, AMBER, and MMHal-Bench benchmarks (with a maximum token length of 512), CEI outperforms state-of-the-art baselines across three LVLMs, with its dynamic variant yielding the lowest overall hallucination rates. By integrating novel mechanistic insights with a scalable intervention, this work advances the mitigation of hallucinations in LVLMs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 18 of its 38 references that are in the map. First posted on arXiv 9 Jan 2026.

Builds on

18 of 38 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  8. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  9. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  10. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
  11. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
  12. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
6 more
  1. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  2. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decodingcited by 118
  3. 2024Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMscited by 49
  4. 2024Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projectioncited by 46
  5. 2025Octopus: Alleviating Hallucination via Dynamic Contrastive Decodingcited by 23
  6. 2025Mitigating Hallucination in Large Vision-Language Models via Adaptive Attention Calibrationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methodscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs
  2. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models
  3. 2026See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
  4. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  5. 2024Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding
  6. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models
  7. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction
  8. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models