Real-Time Visual Attribution Streaming in Thinking Model
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
We present an amortized framework for real-time visual attribution streaming in multimodal thinking models. When these models generate code from a screenshot or solve math problems from images, their long reasoning traces should be grounded in visual evidence. However, verifying this reliance is challenging: faithful causal methods require costly repeated backward passes or perturbations, while raw attention maps offer instant access, they lack causal validity. To resolve this, we introduce an amortized approach that learns to estimate the causal effects of semantic regions directly from the rich signals encoded in attention features. Across five diverse benchmarks and four thinking models, our approach achieves faithfulness comparable to exhaustive causal methods while enabling visual attribution streaming, where users observe grounding evidence as the model reasons, not after. Our results demonstrate that real-time, faithful attribution in multimodal thinking models is achievable through lightweight learning, not brute-force computation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 85 references that are in the map. First posted on arXiv 17 Apr 2026.
Builds on
26 of 85 references.
Closest ancestors · 6
Same direction, newest first
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
- 2025MiMo-VL Technical Report4 Jun
- 2025Argus: Vision-Centric Reasoning with Grounded Chain-of-Thought29 May
- 2025Grounded Chain-of-Thought for Multimodal Large Language Models17 Mar
- 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation14 Jun
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
Other references · 20
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2025DINOv3cited by 1,297
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
- 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
- 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 214
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 159
- 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 126
- 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
8 more
- 2016Interpretation of Prediction Models Using the Input Gradientcited by 24
- 2024LVLM-Interpret: An Interpretability Tool for Large Vision-Language Modelscited by 23
- 2024Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attributioncited by 21
- 2024TLDR: Token-Level Detective Reward Model for Large Vision Language Modelscited by 15
- 2025Learning to Attribute with Attentioncited by 11
- 2024Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models?cited by 8
- 2025GLIMPSE: Holistic Cross-Modal Explainability for Large Vision-Language Modelscited by 5
- 2025Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Modelscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025ISO-Bench: Benchmarking Multimodal Causal Reasoning in Visual-Language Models through Procedural Plans
- 2024Multimodal Causal Reasoning Benchmark: Challenging Vision Large Language Models to Discern Causal Links Across Modalities
- 2026VIG: Visual Information Gain as a Reward Signal for Multimodal Chain-of-Thought Compression
- 2026From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning
- 2026VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning
- 2026Think While Watching: Online Streaming Segment-Level Memory for Multi-Turn Video Reasoning in Multimodal Large Language Models
- 2026TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memory
- 2025CauSight: Learning to Supersense for Visual Causal Discovery