# Shallow-π: Knowledge Distillation for Flow-based VLAs arXiv: https://arxiv.org/abs/2601.20262 · HTML: https://mlsanity.com/paper/2601.20262/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models. - Builds on 41 of its 44 references that are in the map. - First posted on arXiv 28 Jan 2026. ## Placed in - Vision-language-action models / [Efficient VLA inference and adaptation](https://mlsanity.com/vla/efficient-vla-inference-and-adaptation/) ## Builds on (most cited first, 30 of 41) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022) - [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019) - [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022) - [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024) - [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023) - [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024) - [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025) - [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023) - [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023) - [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025) - [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024) - [SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics](https://mlsanity.com/paper/2506.01844/) (2025) - [CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation](https://mlsanity.com/paper/2411.19650/) (2024) - [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021) - [Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation](https://mlsanity.com/paper/2401.02117/) (2024) - [TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12514/) (2024) - [π*0.6: a VLA That Learns From Experience](https://mlsanity.com/paper/2511.14759/) (2025) - [Real-Time Execution of Action Chunking Flow Policies](https://mlsanity.com/paper/2506.07339/) (2025) - [Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation](https://mlsanity.com/paper/2405.07503/) (2024) - [VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model](https://mlsanity.com/paper/2509.09372/) (2025) - [MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices](https://mlsanity.com/paper/2312.16886/) (2023) - [DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution](https://mlsanity.com/paper/2411.02359/) (2024) - [EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models](https://mlsanity.com/paper/2506.10100/) (2025) - [VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching](https://mlsanity.com/paper/2502.02175/) (2025) - [CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification](https://mlsanity.com/paper/2508.21046/) (2025) - [MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation](https://mlsanity.com/paper/2503.20384/) (2025) - [ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training](https://mlsanity.com/paper/2509.01819/) (2025) - [TinyFusion: Diffusion Transformers Learned Shallow](https://mlsanity.com/paper/2412.01199/) (2024) - [SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruning](https://mlsanity.com/paper/2509.05614/) (2025) ## Built on by (most cited first, 8 of 8) - [FASTER: Rethinking Real-Time Flow VLAs](https://mlsanity.com/paper/2603.19199/) (2026) - [SnapFlow: One-Step Action Generation for Flow-Matching VLAs via Progressive Self-Distillation](https://mlsanity.com/paper/2604.05656/) (2026) - [TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM](https://mlsanity.com/paper/2607.27205/) (2026) - [Neuro-Symbolic Safety Guidance for Vision-Language-Action Models via Constrained Flow Matching](https://mlsanity.com/paper/2607.01378/) (2026) - [Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?](https://mlsanity.com/paper/2606.27755/) (2026) - [Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cycle](https://mlsanity.com/paper/2607.22877/) (2026) - [IMLE-VLA: Fast Single-Step Action Generation for Vision-Language-Action Policies](https://mlsanity.com/paper/2609.10915/) (2026) - [Remember What You Did: Action-History Memory with Dual-Expert Denoising for Long-Horizon Vision-Language-Action Policies](https://mlsanity.com/paper/2609.37307/) (2026) ## Abstract (arXiv) The growing demand for real-time robotic deployment necessitates fast and on-device inference for vision-language-action (VLA) models. Within the VLA literature, efficiency has been extensively studied at the token level, such as visual token pruning. In contrast, systematic transformer layer reduction has received limited attention and, to the best of our knowledge, has not been explored for flow-based VLA models under knowledge distillation. In this work, we propose Shallow-pi, a principled knowledge distillation framework that aggressively reduces the transformer depth of both the VLM backbone and the flow-based action head, compressing the model from 18 to 6 layers. Shallow-pi achieves over two times faster inference with less than one percent absolute drop in success rate on standard manipulation benchmarks, establishing state-of-the-art performance among reduced VLA models. Crucially, we validate our approach through industrial-scale real-world experiments on Jetson Orin and Jetson Thor across multiple robot platforms, including humanoid systems, in complex and dynamic manipulation scenarios.