# Anatomy of a Closed-Loop Collapse: A Causal Case Study of a Compressed VLA Policy

arXiv: https://arxiv.org/abs/2609.23048 · HTML: https://mlsanity.com/paper/2609.23048/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 19 Sep 2026.

## Builds on (most cited first, 16 of 16)

- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12514/) (2024)
- [ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills](https://mlsanity.com/paper/2302.04659/) (2023)
- [AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real World](https://mlsanity.com/paper/2503.24278/) (2025)
- [CEED-VLA: Consistency Vision-Language-Action Model with Early-Exit Decoding](https://mlsanity.com/paper/2506.13725/) (2025)
- [The Pitfalls of Imitation Learning when Actions are Continuous](https://mlsanity.com/paper/2503.09722/) (2025)
- [RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models](https://mlsanity.com/paper/2506.17639/) (2025)
- [Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models](https://mlsanity.com/paper/2603.19233/) (2026)
- [VLA-Trace: Diagnosing Vision-Language-Action Models through Representation and Behavior Tracing](https://mlsanity.com/paper/2605.30117/) (2026)
- [Guiding Data Collection via Factored Scaling Curves](https://mlsanity.com/paper/2505.07728/) (2025)
- [VITA-VLA: Efficiently Teaching Vision-Language Models to Act via Action Expert Distillation](https://mlsanity.com/paper/2510.09607/) (2025)
- [Learning Actionable Manipulation Recovery via Counterfactual Failure Synthesis](https://mlsanity.com/paper/2603.13528/) (2026)
- [SilentDrift: Exploiting Action Chunking for Stealthy Backdoor Attacks on Vision-Language-Action Models](https://mlsanity.com/paper/2601.14323/) (2026)
- [Disambiguate Gripper State in Grasp-Based Tasks: Pseudo-Tactile as Feedback Enables Pure Simulation Learning](https://mlsanity.com/paper/2503.23835/) (2025)
- [Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation](https://mlsanity.com/paper/2605.16241/) (2026)

## Abstract (arXiv)

Compressed manipulation policies can pass offline evaluation while failing in closed-loop execution; this dissociation is established in prior work and is not our claim. We contribute a causal anatomy of one naturally occurring case. An 8-layer distillation of Octo-Base retains 86% of parameters, passes every offline check we applied (0.996 and 1.000 teacher-ratios on the family's own validation metrics), and collapses in closed loop: 0/72 vs. the teacher's 40/72 on a simulated WidowX pick-and-place task. The collapse is structured, not diffuse: early task stages degrade gradually (the student moves the object at 90% of the teacher's rate and grasps at 55%), while transport-to-target fails categorically, at 0% in every training variant. Paired action-trace forensics isolate the signature: a negative, late-heavy z residual, roughly 10x its post-repair magnitude, and persistent across the base distillation and both continuation branches. Four standard therapies fail under matched controls: continued training and in-domain offline data leave success at zero, even though the latter measurably improves marginal action statistics; command-level compensation recovers nothing at any offset, although the same perturbations degrade healthy policies; clamping the symptom in the command channel preserves grasping, yet success stays at floor. A minimal-pair intervention that substitutes half of the training stream with deployment-distribution teacher rollouts, with every other setting held fixed, restores parity with the teacher (18/36 vs. 17/36 held-out), eliminates that signature, and recovers a teacher-like perturbation-response profile. We claim existence, not universality. Operationally, offline gates, including a family's own validation metrics, are insufficient acceptance tests for compressed policies; a few dozen closed-loop trials sufficed to find what they missed.