mlsanityData to 7 Oct 2026

Circuit Tracing in Vision-Language Models: Understanding the Internal Mechanisms of Multimodal Thinking

arXiv 2602.20330 ↗First posted 23 Feb 2026cs.CV cs.AI cs.LGcited by 6

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Vision-language models (VLMs) are powerful but remain opaque black boxes. We introduce the first framework for transparent circuit tracing in VLMs to systematically analyze multimodal reasoning. By utilizing transcoders, attribution graphs, and attention-based methods, we uncover how VLMs hierarchically integrate visual and semantic concepts. We reveal that distinct visual feature circuits can handle mathematical reasoning and support cross-modal associations. Validated through feature steering and circuit patching, our framework proves these circuits are causal and controllable, laying the groundwork for more explainable and reliable VLMs.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 40 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

20 of 40 references.

References · 20

  1. 2024GPT-4o System Cardcited by 4,467
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2017Decoupled Weight Decay Regularizationcited by 1,722
  4. 2025Gemma 3 Technical Reportcited by 1,686
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  6. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  7. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  8. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  9. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  10. 2022Mass-Editing Memory in a Transformercited by 976
  11. 2022Toy Models of Superpositioncited by 928
  12. 2019Attention is not Explanationcited by 916
8 more
  1. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  2. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  3. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 307
  4. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 186
  5. 2023Localizing Model Behavior with Path Patchingcited by 182
  6. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106
  7. 2022Polysemanticity and Capacity in Neural Networkscited by 74
  8. 2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Mechanistically Interpreting Compression in Vision-Language Modelscited by 0
  2. 2026Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Modelscited by 0
  3. 2026Brand-as-Memory: Vision-Language Models Encode Causal, Mechanistically Localizable Credibility Priors for News Sourcescited by 0
  4. 2026CARD: Diagnosing Belief to Action Routing Failures in Vision Language Modelscited by 0
  5. 2026Circuit-Diff: Factual Edit-based Intervention Method for Localizing Knowledge in Attribution Graphscited by 0
  6. 2026FLIP: Final Layer Inference-Time Probing for Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoning
  2. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
  3. 2025Diagnosing Visual Reasoning: Challenges, Insights, and a Path Forward
  4. 2025Multimodal LLM Augmented Reasoning for Interpretable Visual Perception Analysis
  5. 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
  6. 2026Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap
  7. 2025MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models
  8. 2025Forgotten Polygons: Multimodal Large Language Models are Shape-Blind