mlsanityData to 14 Oct 2026

SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering

arXiv 2607.04163 ↗First posted 5 Jul 2026cs.CV cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large Vision-Language Models (LVLMs) have achieved remarkable progress in visual understanding tasks such as image captioning and visual question answering. However, they remain susceptible to hallucinations, generating content that is inconsistent with the actual visual input. Existing methods primarily intervene at the decoding stage, while overlooking a critical source of hallucinations: irrelevant or noisy visual tokens that mislead the decoding process. To address this issue, we propose SeeMe, a training-free framework that introduces the concept of feature engineering from traditional machine learning into LVLMs. SeeMe restructures visual tokens through a three-stage token engineering process to suppress hallucination sources while preserving informative visual evidence. Experiments on MME, POPE, and AMBER benchmarks across four LVLMs demonstrate that SeeMe consistently reduces hallucinations and improves output consistency, providing a novel perspective for mitigating hallucinations in LVLMs.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 53 references that are in the map. First posted on arXiv 5 Jul 2026.

Builds on

45 of 53 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Mitigating Hallucinations in Vision-Language Models through Image-Guided Head Suppression22 May
  2. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models18 May
  3. 2024Multi-Object Hallucination in Vision-Language Models8 Jul
  4. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention18 Jun
  5. 2024Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models28 May
  6. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback22 Apr
  7. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding28 Feb
  8. 2024A Survey on Hallucination in Large Vision-Language Models1 Feb
  9. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites4 Dec
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  11. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  12. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
4 more
  1. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  2. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  3. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  4. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  7. 2023Qwen Technical Reportcited by 3,344
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  10. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  11. 2022Token Merging: Your ViT But Fastercited by 1,021
  12. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
17 more
  1. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  2. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  3. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  4. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  5. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  6. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  7. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  8. 2023Token Fusion: Bridging the Gap between Token Pruning and Token Mergingcited by 125
  9. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 125
  10. 2022OpenFE: Automated Feature Generation with Expert-level Performancecited by 52
  11. 2026Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoringcited by 21
  12. 2026PointCoT: A Multi-modal Benchmark for Explicit 3D Geometric Reasoningcited by 19
  13. 2022Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problemcited by 18
  14. 2026Not All Queries Need Deep Thought: CoFiCot for Adaptive Coarse-to-fine Stateful Refinementcited by 13
  15. 2023InfMLLM: A Unified Framework for Visual-Language Taskscited by 11
  16. 2025Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoningcited by 11
  17. 2025STAR: Stage-Wise Attention-Guided Token Reduction for Efficient Large Vision-Language Models Inferencecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoningcited by 2
  2. 2026SPARK: Susceptibility-Guided Profiling and Steering of Latent Reasoning States in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  2. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
  3. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction
  4. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations
  5. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning
  6. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
  7. 2025IKOD: Mitigating Visual Attention Degradation in Large Vision-Language Models
  8. 2025MAP: Mitigating Hallucinations in Large Vision-Language Models with Map-Level Attention Processing