From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodability
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
How much of a vision-language model's (VLM) spatial understanding remains after the action post-training process of building a vision-language-action model (VLA)? We probe depth perception, a primitive of spatiogeometric understanding, from every decoder layer of a weight-matched open-source base VLM/VLA pair: Molmo2-ER and MolmoAct2-LIBERO. First, the VLA decodes depth worse at every layer, a persistent gap we call the floor. Second, the degradation is not uniform: while the base VLM's depth decodability improves through its final layers, the VLA's collapses, an additional late-layer drop we call the cliff. We causally localize the cliff to late-layer MLP interference: ablating the late-layer MLP writes recovers the majority of the terminal decodability cliff, while matched attention ablations and the same intervention in the weight-matched base VLM produce no comparable recovery. A module-level decomposition explains this dissociation: the base VLM carries depth most accessibly in accumulated MLP writes, whereas action post-training collapses depth decodability in the late accumulated writes.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 21 references that are in the map. First posted on arXiv 9 Aug 2026.
Builds on
17 of 21 references.
Closest ancestors · 2
Same direction, newest first
- 2025Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization29 Oct
- 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better29 May
Other references · 15
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2021Vision Transformers for Dense Predictioncited by 1,803
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
- 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
- 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
- 2024Probing the 3D Awareness of Visual Foundation Modelscited by 177
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning
- 2026Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning
- 2025Hidden in plain sight: VLMs overlook their visual representations
- 2026Revisiting Parameter Redundancy in Vision-Language-Action Models: Insights from VLM-to-VLA Adaptation
- 2026From Edges to Depth: Probing the Spatial Hierarchy in Vision Transformers
- 2025LAST: LeArning to Think in Space and Time for Generalist Vision-Language Models
- 2026When Depth Is Better Told Than Shown: Depth-Ordinal Prompting for Vision-Language Spatial Reasoning
- 2025Igniting VLMs toward the Embodied Space