mlsanityData to 14 Oct 2026

EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception

arXiv 2609.37123 ↗First posted 29 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning also fits: Visual token pruning and efficiency

Abstract (arXiv)

Fine-grained visual perception enables vision-language models to distinguish subtle attributes and ground their answers in visual evidence. In high-resolution scenes, processing the whole image at greater resolution spends visual tokens on irrelevant content, while isolated crops can lose the context needed to interpret the selected evidence. We introduce EviViT, a lightweight attachment that learns where a pretrained vision transformer should acquire detail. Human visual-search traces supervise a question-conditioned evidence density, which guides regional re-reading from the original pixels and the allocation of visual tokens. A sparse, coordinate-aware bridge then connects the regional features to the global scene, allowing the host to interpret precise evidence in context. Learned with the host backbone frozen, the attachment serves both the base model and compatible post-trained descendants without refitting. Experiments across nine hosts show consistent gains in average fine-grained accuracy. Matched-budget comparisons further show that EviViT outperforms global-only processing at every tested token ceiling while using fewer visual tokens.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 33 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

31 of 33 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Self-Prophetic Decoding to Unlock Visual Search in LVLMs27 May
  2. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search9 Sep
  3. 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoning8 Aug
  4. 2025FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering26 Jun
  5. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?29 Mar
  6. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  8. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  9. 2022Token Merging: Your ViT But Fastercited by 1,021
  10. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  11. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  12. 2022Vision Transformer with Deformable Attentioncited by 295
13 more
  1. 2016Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?cited by 111
  2. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 77
  3. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 50
  4. 2025VisRL: Intention-Driven Visual Perception via Reinforced Reasoningcited by 39
  5. 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisitioncited by 21
  6. 2021VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answeringcited by 16
  7. 2026Vero: An Open RL Recipe for General Visual Reasoningcited by 11
  8. 2026OPD-V: Visual On-Policy Self-Distillation with Modality Balancecited by 8
  9. 20264DThinker: Thinking with 4D Imagery for Dynamic Spatial Understandingcited by 4
  10. 2026PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Modelscited by 4
  11. 2026Focus-Scan-Refine: From Human Visual Perception to Efficient Visual Token Pruningcited by 3
  12. 2026OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruningcited by 3
  13. 2026PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seekingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EviAnchor: Mitigating Hallucinations in Large Vision-Language Models via Regional Visual Evidence Compensation
  2. 2026DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models
  3. 2026Improving Visual Reasoning with Iterative Evidence Refinement
  4. 2026E2S-Pruner: Progressive Two-Stage Evidence Fusion for Visual Token Pruning in Vision-Language Models
  5. 2026Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning
  6. 2026Foveated Reasoning: Stateful, Action-based Visual Focusing for Vision-Language Models
  7. 2026LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models
  8. 2026SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference