mlsanityData to 14 Oct 2026

LOCUS: Local Visual Cue Search for Enhancing Fine-Grained Perception in Multimodal Large Language Models

arXiv 2606.16586 ↗First posted 15 Jun 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) remain unreliable on fine-grained visual perception, even when high-resolution inputs preserve the necessary local details. We identify this limitation as visual context rot: decisive evidence may exist in the full image, yet fail to be reliably selected and used amid redundant visual context. We propose LOCUS (LOcal visual CUe Search), a training framework that teaches MLLMs to internalize local evidence search through a verifiable proxy task. During training, LOCUS provides a local crop as a visual cue and optimizes the model to recover its spatial support in the full image using an IoU-based reward. The visual cue is used only during training, leaving the standard image-question inference interface unchanged. Experiments across fine-grained perception, hallucination, general understanding, and reasoning benchmarks show that LOCUS improves localization-sensitive visual understanding while preserving broad capabilities. Attention analyses further indicate stronger focus on task-relevant evidence regions, suggesting that training-time visual cue search provides an effective route to internalized fine-grained evidence selection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 45 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

28 of 45 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception12 Feb
  2. 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling28 Sep
  3. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?23 Aug
  4. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs24 Jun

Other references · 23

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  6. 2016Modeling Context in Referring Expressionscited by 1,063
  7. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  8. 2016A Diagram Is Worth A Dozen Imagescited by 804
  9. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  10. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  11. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  12. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
11 more
  1. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  2. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
  3. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 286
  4. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
  5. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  6. 2025Visual Jigsaw Post-Training Improves MLLMscited by 26
  7. 2025CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimizationcited by 26
  8. 2025ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMscited by 25
  9. 2023Towards Perceiving Small Visual Details in Zero-shot Visual Question Answering with Multimodal LLMscited by 15
  10. 2026How Vision Becomes Language: A Layer-wise Information-Theoretic Analysis of Multimodal Reasoningcited by 8
  11. 2026When Thinking Hurts: Mitigating Visual Forgetting in Video Reasoning via Frame Repetitioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models
  2. 2026VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes
  3. 2026LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
  4. 2026MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
  5. 2026Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning
  6. 2026ReMAP: Restoring the Perceptual Cycle with Reasoning-Time Latent Visual Memory
  7. 2025Visual Funnel: Resolving Contextual Blindness in Multimodal Large Language Models
  8. 2026DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues