# Mechanistic interpretability for steering vision-language-action models

arXiv: https://arxiv.org/abs/2509.00328 · HTML: https://mlsanity.com/paper/2509.00328/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models.
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 30 Aug 2025.

## Placed in

- Vision-language-action models / [Generalist VLA models and benchmarks](https://mlsanity.com/vla/generalist-vla-models-and-benchmarks/)
- Mechanistic interpretability / [Steering and probing for LLM safety](https://mlsanity.com/mechinterp/steering-and-probing-for-llm-safety/)

## Builds on (most cited first, 15 of 15)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [Auditing language models for hidden objectives](https://mlsanity.com/paper/2503.10965/) (2025)
- [A Survey on Interpretable Reinforcement Learning](https://mlsanity.com/paper/2112.13112/) (2021)
- [Understanding the Dependence of Perception Model Competency on Regions in an Image](https://mlsanity.com/paper/2407.10543/) (2024)

## Built on by (most cited first, 30 of 31)

- [Observing and Controlling Features in Vision-Language-Action Models](https://mlsanity.com/paper/2603.05487/) (2026)
- [Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models](https://mlsanity.com/paper/2603.19183/) (2026)
- [Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations](https://mlsanity.com/paper/2511.22697/) (2025)
- [Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models](https://mlsanity.com/paper/2603.19233/) (2026)
- [An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges](https://mlsanity.com/paper/2512.11362/) (2025)
- [Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Models](https://mlsanity.com/paper/2605.29074/) (2026)
- [Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering](https://mlsanity.com/paper/2606.12299/) (2026)
- [Contrastive Conceptor Activation Steering (COAST): Unlocking Vision-Language-Action Models through Hidden States](https://mlsanity.com/paper/2605.17144/) (2026)
- [Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control](https://mlsanity.com/paper/2607.14943/) (2026)
- [Decoding Task Progress from VLA Representations](https://mlsanity.com/paper/2608.13474/) (2026)
- [Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Models](https://mlsanity.com/paper/2605.00321/) (2026)
- [Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies](https://mlsanity.com/paper/2605.17204/) (2026)
- [Early Warning Signals for OpenVLA Failure under Visual Distribution Shift](https://mlsanity.com/paper/2606.29699/) (2026)
- [Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering](https://mlsanity.com/paper/2606.29201/) (2026)
- [Present but Not Remembered: Auditing How Frozen VLAs Encode, Deploy, and Steer Visual History](https://mlsanity.com/paper/2607.03372/) (2026)
- [DiMaS: Distribution Matching for Steering Vision-Language-Action Models](https://mlsanity.com/paper/2607.14280/) (2026)
- [What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning](https://mlsanity.com/paper/2607.16938/) (2026)
- [Closed-Loop Neural Activation Control in Vision-Language-Action Models](https://mlsanity.com/paper/2606.00269/) (2026)
- [Act on What You See: Unlocking Safe Social Navigation in Vision-Language-Action Models](https://mlsanity.com/paper/2606.10495/) (2026)
- [Multimodal Model Diffing for Feature Discovery and Control](https://mlsanity.com/paper/2608.09928/) (2026)
- [FabriMAE I Trust Myself? Self-Evaluating VLA Action Generation with Markov Attention Entropy](https://mlsanity.com/paper/2608.16697/) (2026)
- [IMPACT-VLA: Interaction-aware Multimodal Propagation Attribution via Counterfactual Trajectories for Vision-Language-Action Policies](https://mlsanity.com/paper/2609.15005/) (2026)
- [What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior](https://mlsanity.com/paper/2609.24576/) (2026)
- [ActGaze: Learning Action-Grounded Gaze through Counterfactual Visual Interventions for High-Precision Manipulation](https://mlsanity.com/paper/2609.28955/) (2026)
- [The Linear Representation Hypothesis for Vision-Language-Action Models](https://mlsanity.com/paper/2609.30996/) (2026)
- [GT-VLA: Target-Conditioned Trace Guidance for Generalizable Robotic Manipulation](https://mlsanity.com/paper/2609.31904/) (2026)
- [The Low-Rank Structure of VLA Reinforcement Learning](https://mlsanity.com/paper/2609.34599/) (2026)
- [Natural State-Prediction Accuracy can Hide Weak Controlled Responsiveness in VLA Readouts](https://mlsanity.com/paper/2609.34684/) (2026)
- [When Instructions Retrieve Trajectories: Diagnosing and Mitigating Generalization Failures in VLA Models](https://mlsanity.com/paper/2609.39971/) (2026)
- [eRLT: Efficient VLA Reinforcement Learning via Action-Relevant Token Routing](https://mlsanity.com/paper/2610.00913/) (2026)

## Abstract (arXiv)

Vision-Language-Action (VLA) models are a promising path to realizing generalist embodied agents that can quickly adapt to new tasks, modalities, and environments. However, methods for interpreting and steering VLAs fall far short of classical robotics pipelines, which are grounded in explicit models of kinematics, dynamics, and control. This lack of mechanistic insight is a central challenge for deploying learned policies in real-world robotics, where robustness and explainability are critical. Motivated by advances in mechanistic interpretability for large language models, we introduce the first framework for interpreting and steering VLAs via their internal representations, enabling direct intervention in model behavior at inference time. We project feedforward activations within transformer layers onto the token embedding basis, identifying sparse semantic directions - such as speed and direction - that are causally linked to action selection. Leveraging these findings, we introduce a general-purpose activation steering method that modulates behavior in real time, without fine-tuning, reward signals, or environment interaction. We evaluate this method on two recent open-source VLAs, Pi0 and OpenVLA, and demonstrate zero-shot behavioral control in simulation (LIBERO) and on a physical robot (UR5). This work demonstrates that interpretable components of embodied VLAs can be systematically harnessed for control - establishing a new paradigm for transparent and steerable foundation models in robotics.