# EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception

arXiv: https://arxiv.org/abs/2609.37123 · HTML: https://mlsanity.com/paper/2609.37123/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 31 of its 33 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2403.16999/) (2024)
- [Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search](https://mlsanity.com/paper/2509.07969/) (2025)
- [Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?](https://mlsanity.com/paper/1606.03556/) (2016)
- [Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation](https://mlsanity.com/paper/2605.18740/) (2026)
- [Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception](https://mlsanity.com/paper/2602.11858/) (2026)
- [VisRL: Intention-Driven Visual Perception via Reinforced Reasoning](https://mlsanity.com/paper/2503.07523/) (2025)
- [SIFThinker: Spatially-Aware Image Focus for Visual Reasoning](https://mlsanity.com/paper/2508.06259/) (2025)
- [AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition](https://mlsanity.com/paper/2512.03794/) (2025)
- [FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering](https://mlsanity.com/paper/2506.21710/) (2025)
- [VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answering](https://mlsanity.com/paper/2109.13116/) (2021)
- [Vero: An Open RL Recipe for General Visual Reasoning](https://mlsanity.com/paper/2604.04917/) (2026)
- [OPD-V: Visual On-Policy Self-Distillation with Modality Balance](https://mlsanity.com/paper/2608.05131/) (2026)
- [4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding](https://mlsanity.com/paper/2605.05997/) (2026)
- [PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models](https://mlsanity.com/paper/2607.24957/) (2026)
- [Focus-Scan-Refine: From Human Visual Perception to Efficient Visual Token Pruning](https://mlsanity.com/paper/2602.05809/) (2026)
- [OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning](https://mlsanity.com/paper/2605.29657/) (2026)
- [PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking](https://mlsanity.com/paper/2607.00115/) (2026)

## Abstract (arXiv)

Fine-grained visual perception enables vision-language models to distinguish subtle attributes and ground their answers in visual evidence. In high-resolution scenes, processing the whole image at greater resolution spends visual tokens on irrelevant content, while isolated crops can lose the context needed to interpret the selected evidence. We introduce EviViT, a lightweight attachment that learns where a pretrained vision transformer should acquire detail. Human visual-search traces supervise a question-conditioned evidence density, which guides regional re-reading from the original pixels and the allocation of visual tokens. A sparse, coordinate-aware bridge then connects the regional features to the global scene, allowing the host to interpret precise evidence in context. Learned with the host backbone frozen, the attachment serves both the base model and compatible post-trained descendants without refitting. Experiments across nine hosts show consistent gains in average fine-grained accuracy. Matched-budget comparisons further show that EviViT outperforms global-only processing at every tested token ceiling while using fewer visual tokens.