mlsanityData to 7 Oct 2026

LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models

arXiv 2603.14882 ↗First posted 16 Mar 2026cs.CVcited by 6

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Vision-Language Models (VLMs) typically assume a uniform spatial fidelity across the entire field of view of visual inputs, dedicating equal precision to even the uninformative regions. By contrast, human vision is neither uniform nor static; it is adaptive, selective, and resource-efficient. In light of this, we present the first systematic analysis of bio-inspired visual representation methods, providing insights for more efficient and adaptive VLMs. We propose LLMind (Looking Like the Mind), a novel training-free framework that mimics foveated encoding and cortical magnification in human vision to achieve adaptive, efficient representations for VLMs under tight pixel budgets. Our key idea is to explore a Bio-inspired Adaptive Sampling Strategy (BASS), enabling a Mobius-parameterized module that performs non-uniform sampling while preserving global scene structure. On top of BASS, we introduce closed-loop semantic feedback (CSF) via test-time adaptation to align perceptual saliency with textual information from the frozen VLM. We evaluate LLMind against uniform and other sampling baselines across diverse scene-level and region-guided visual question answering benchmarks. The results show dramatic gains, with average improvements of +20% on VQAv2, +38% on Seed-Bench, and +37% on A-OKVQA compared to uniform sampling under tight pixel budgets. More surprisingly, LLMind retains up to 82%, 92%, and 97% of the full-resolution performance using only 1%, 3%, and 5% of the pixels, respectively. Moreover, LLMind is lightweight, plug-and-play, and compatible with existing VLMs without requiring architectural changes.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 58 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

36 of 58 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025SmolVLM: Redefining small and efficient multimodal models7 Apr

Other references · 35

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  9. 2022Visual Prompt Tuningcited by 1,674
  10. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  11. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  12. 2021Transformers in Vision: A Surveycited by 1,009
23 more
  1. 2022Token Merging: Your ViT But Fastercited by 995
  2. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  3. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  4. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  5. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  6. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  7. 2020A Survey on Visual Transformercited by 653
  8. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  9. 2024FeatUp: A Model-Agnostic Framework for Features at Any Resolutioncited by 114
  10. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  11. 2022A General Survey on Attention Mechanisms in Deep Learningcited by 64
  12. 2024ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Modelscited by 33
  13. 2022Peripheral Vision Transformercited by 27
  14. 2020Emergent Properties of Foveated Perceptual Systemscited by 22
  15. 2014Object Detection Through Exploration With A Foveated Visual Fieldcited by 14
  16. 2025Emulating Human-like Adaptive Vision for Efficient and Flexible Machine Visual Perceptioncited by 13
  17. 2017Central and peripheral vision for scene recognition: A neurocomputational modeling explorationcited by 8
  18. 2018Human peripheral blur is optimal for object recognitioncited by 8
  19. 2022Behind the Machine's Gaze: Neural Networks with Biologically-inspired Constraints Exhibit Human-like Visual Attentioncited by 7
  20. 2023Foveation in the Era of Deep Learningcited by 7
  21. 2025Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspectivecited by 7
  22. 2024ScribGen: Generating Scribble Art Through Metaheuristicscited by 3
  23. 2023Learning to search for and detect objects in foveal images using deep learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Activating Distributed Visual Region within LLMs for Efficient and Effective Vision-Language Training and Inference
  2. 2025Adapting Vision-Language Models Without Labels: A Comprehensive Survey
  3. 2026Language-Instructed Vision Embeddings for Controllable and Generalizable Perception
  4. 2025Rethinking Visual Information Processing in Multimodal LLMs
  5. 2025Enhancing Self-Supervised Visual Representation Learning via Low-Rank Adapted LLMs
  6. 2024Frustratingly Easy Test-Time Adaptation of Vision-Language Models
  7. 2025Vision Language Models: A Survey of 26K Papers
  8. 2026PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference