mlsanityData to 7 Oct 2026

Decoding the Pulse of Reasoning VLMs in Multi-Image Understanding Tasks

arXiv 2603.04676 ↗First posted 4 Mar 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multi-image reasoning remains a significant challenge for vision-language models (VLMs). We investigate a previously overlooked phenomenon: during chain-of-thought (CoT) generation, the text-to-image (T2I) attention of reasoning VLMs exhibits diffuse "pulses": sporadic and unfocused attention patterns that fail to concentrate on task-relevant images. We further reveal a systematic positional bias in attention allocation across images. Motivated by these observations, we propose PulseFocus, a training-free, inference-time method that structures CoT reasoning into interleaved plan/focus blocks with soft attention gating. By forcing the model to explicitly plan which image to examine and then gating decode-time attention to the referenced image, PulseFocus sharpens attention focus and yields consistent improvements on multi-image benchmarks like BLINK benchmark (+3.7%) and MuirBench (+1.07%).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 20 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

19 of 20 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Rethinking Causal Mask Attention for Vision-Language Inference24 May
  2. 2025CMMCoT: Enhancing Complex Multi-Image Comprehension via Multi-Modal Chain-of-Thought and Memory Augmentation7 Mar
  3. 2023IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models24 May

Other references · 16

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2025OpenAI GPT-5 System Cardcited by 910
  5. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  6. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  7. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 178
  8. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
  9. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 61
  10. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmarkcited by 53
  11. 2024MIBench: Evaluating Multimodal Large Language Models over Multiple Imagescited by 36
  12. 2024V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encodingcited by 28
4 more
  1. 2025Interleaved Reasoning for Large Language Models via Reinforcement Learningcited by 21
  2. 2026More Images, More Problems? A Controlled Analysis of VLM Failure Modescited by 7
  3. 2024Control Large Language Models via Divide and Conquercited by 5
  4. 2024Multi-layer Learnable Attention Mask for Multimodal Taskscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning
  2. 2025Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning
  3. 2025SIFThinker: Spatially-Aware Image Focus for Visual Reasoning
  4. 2025CoFFT: Chain of Foresight-Focus Thought for Visual Language Models
  5. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
  6. 2026From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning
  7. 2026Visual Access Boundaries in Vision-Language Model Reasoning
  8. 2025BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception