mlsanityData to 7 Oct 2026

SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination

arXiv 2512.07730 ↗First posted 8 Dec 2025cs.CV cs.AIcited by 3

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Although Multimodal Large Language Models (MLLMs) have advanced substantially, they remain vulnerable to object hallucination caused by language priors and visual information loss. To address this, we propose SAVE (Sparse Autoencoder-Driven Visual Information Enhancement), a framework that mitigates hallucination by steering the model along Sparse Autoencoder (SAE) latent features. A binary object-presence question-answering probe identifies the SAE features most indicative of the model's visual information processing, referred to as visual understanding features. Steering the model along these identified features reinforces grounded visual understanding and effectively reduces hallucination. With its simple design, SAVE outperforms state-of-the-art training-free methods on standard benchmarks, achieving a 10%p improvement in CHAIRS and consistent gains on POPE and MMHal-Bench. Extensive evaluations across multiple models and layers confirm the robustness and generalizability of our approach. Further analysis reveals that steering along visual understanding features suppresses the generation of uncertain object tokens and increases attention to image tokens, mitigating hallucination. Code is released at https://github.com/wiarae/SAVE.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 45 references that are in the map. First posted on arXiv 8 Dec 2025.

Builds on

32 of 45 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding23 May
  2. 2025Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding1 Mar
  3. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering5 Feb
  4. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  5. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation15 Oct
  6. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations3 Oct
  7. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models4 Aug
  8. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  9. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  10. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding28 Feb
  11. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  12. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
2 more
  1. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  2. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 18

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  7. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  8. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  9. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  10. 2018Object Hallucination in Image Captioningcited by 661
  11. 2024Scaling and evaluating sparse autoencoderscited by 530
  12. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
6 more
  1. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 127
  2. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  3. 2024Sparse autoencoders reveal selective remapping of visual concepts during adaptationcited by 66
  4. 2024Large Multi-modal Models Can Interpret Features in Large Multi-modal Modelscited by 23
  5. 2025SAE-V: Interpreting Multimodal Models for Enhanced Alignmentcited by 22
  6. 2025Line of Sight: On Linear Representations in VLLMscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HalluSAE: Detecting Hallucinations in Large Language Models via Sparse Auto-Encoderscited by 3
  2. 2026TORINO: Token Reduction via Interpretable Concept Overlap in Vision-Language Modelscited by 0
  3. 2026NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding
  2. 2025Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation
  3. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training
  4. 2025Exposing Hallucinations To Suppress Them: VLMs Representation Editing With Generative Anchors
  5. 2026VCE: A zero-cost hallucination mitigation method of LVLMs via visual contrastive editing
  6. 2026Rethinking Visual Neglect: Steering via Context-Preference for MLLM Hallucination Mitigation
  7. 2025CAI: Caption-Sensitive Attention Intervention for Mitigating Object Hallucination in Large Vision-Language Models
  8. 2026DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing