# VisualScratchpad: Inference-time Visual Concepts Analysis in Vision Language Models

arXiv: https://arxiv.org/abs/2603.07335 · HTML: https://mlsanity.com/paper/2603.07335/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 7 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 23 of 23)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2401.06209/) (2024)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Open Problems in Mechanistic Interpretability](https://mlsanity.com/paper/2501.16496/) (2025)
- [A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders](https://mlsanity.com/paper/2409.14507/) (2024)
- [BatchTopK Sparse Autoencoders](https://mlsanity.com/paper/2412.06410/) (2024)
- [Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models](https://mlsanity.com/paper/2502.12892/) (2025)
- [Sparse autoencoders reveal selective remapping of visual concepts during adaptation](https://mlsanity.com/paper/2412.05276/) (2024)
- [NSML: A Machine Learning Platform That Enables You to Focus on Your Models](https://mlsanity.com/paper/1712.05902/) (2017)
- [SAE-V: Interpreting Multimodal Models for Enhanced Alignment](https://mlsanity.com/paper/2502.17514/) (2025)
- [VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set](https://mlsanity.com/paper/2510.21323/) (2025)
- [CytoSAE: Interpretable Cell Embeddings for Hematology](https://mlsanity.com/paper/2507.12464/) (2025)
- [ConceptScope: Characterizing Dataset Bias via Disentangled Visual Concepts](https://mlsanity.com/paper/2510.26186/) (2025)
- [CFM: Language-aligned Concept Foundation Model for Vision](https://mlsanity.com/paper/2601.13798/) (2026)
- [ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Models](https://mlsanity.com/paper/2509.20376/) (2025)

## Built on by (most cited first, 1 of 1)

- [Grounding Isn't Knowing: Do VLMs Need Object Localization for Spatial Reasoning?](https://mlsanity.com/paper/2608.23074/) (2026)

## Abstract (arXiv)

High-performing vision language models still produce incorrect answers, yet their failure modes are often difficult to explain. To make model internals more accessible and enable systematic debugging, we introduce VisualScratchpad, an interactive interface for visual concept analysis during inference. We apply sparse autoencoders to the vision encoder and link the resulting visual concepts to text tokens via text-to-image attention, allowing us to examine which visual concepts are both captured by the vision encoder and utilized by the language model. VisualScratchpad also provides a token-latent heatmap view that suggests a sufficient set of latents for effective concept ablation in causal analysis. Through case studies, we reveal three underexplored failure modes: limited cross-modal alignment, misleading visual concepts, and unused hidden cues. Project page: https://hyesulim.github.io/visual_scratchpad_projectpage/