# SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration

arXiv: https://arxiv.org/abs/2506.12723 · HTML: https://mlsanity.com/paper/2506.12723/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models.
- Builds on 35 of its 43 references that are in the map.
- First posted on arXiv 15 Jun 2025.

## Placed in

- Vision-language-action models / [Efficient VLA inference and adaptation](https://mlsanity.com/vla/efficient-vla-inference-and-adaptation/)
- Efficient LLM inference and serving / [Visual token pruning in multimodal models](https://mlsanity.com/inference/visual-token-pruning-in-multimodal-models/)

## Builds on (most cited first, 30 of 35)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation](https://mlsanity.com/paper/2410.07864/) (2024)
- [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025)
- [An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models](https://mlsanity.com/paper/2403.06764/) (2024)
- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation](https://mlsanity.com/paper/2411.19650/) (2024)
- [LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models](https://mlsanity.com/paper/2403.15388/) (2024)
- [SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference](https://mlsanity.com/paper/2410.04417/) (2024)
- [VisionZip: Longer is Better but Not Necessary in Vision Language Models](https://mlsanity.com/paper/2412.04467/) (2024)
- [Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models](https://mlsanity.com/paper/2502.19417/) (2025)
- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [Cache Me if You Can: Accelerating Diffusion Models through Block Caching](https://mlsanity.com/paper/2312.03209/) (2023)
- [Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching](https://mlsanity.com/paper/2406.01733/) (2024)
- [DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution](https://mlsanity.com/paper/2411.02359/) (2024)
- [EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models](https://mlsanity.com/paper/2506.10100/) (2025)
- [PD-VLA: Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decoding](https://mlsanity.com/paper/2503.02310/) (2025)
- [VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching](https://mlsanity.com/paper/2502.02175/) (2025)
- [FastVLM: Efficient Vision Encoding for Vision Language Models](https://mlsanity.com/paper/2412.13303/) (2024)
- [HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers](https://mlsanity.com/paper/2410.05273/) (2024)
- [Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance](https://mlsanity.com/paper/2203.08368/) (2022)
- [A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM](https://mlsanity.com/paper/2410.15549/) (2024)
- [Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Control](https://mlsanity.com/paper/2412.01034/) (2024)
- [Retraining-free Model Quantization via One-Shot Weight-Coupling Learning](https://mlsanity.com/paper/2401.01543/) (2024)

## Built on by (most cited first, 30 of 58)

- [Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2508.13073/) (2025)
- [SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruning](https://mlsanity.com/paper/2509.05614/) (2025)
- [The Better You Learn, The Smarter You Prune: Towards Efficient Vision-language-action Models via Differentiable Token Pruning](https://mlsanity.com/paper/2509.12594/) (2025)
- [Action-aware Dynamic Pruning for Efficient Vision-Language-Action Manipulation](https://mlsanity.com/paper/2509.22093/) (2025)
- [A Survey on Efficient Vision-Language-Action Models](https://mlsanity.com/paper/2510.24795/) (2025)
- [Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey](https://mlsanity.com/paper/2510.17111/) (2025)
- [AVA-VLA: Improving Vision-Language-Action models with Active Visual Attention](https://mlsanity.com/paper/2511.18960/) (2025)
- [RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models](https://mlsanity.com/paper/2506.17639/) (2025)
- [FASTER: Rethinking Real-Time Flow VLAs](https://mlsanity.com/paper/2603.19199/) (2026)
- [Bridging the Semantic-Action Gap in Visual Token Pruning for Efficient VLA Inference](https://mlsanity.com/paper/2511.16449/) (2025)
- [SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models](https://mlsanity.com/paper/2509.09090/) (2025)
- [Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards](https://mlsanity.com/paper/2510.20867/) (2025)
- [VGGT-DP: Generalizable Robot Control via Vision Foundation Models](https://mlsanity.com/paper/2509.18778/) (2025)
- [GeoSem-WAM: Geometry- and Semantic-Aware World Action Models](https://mlsanity.com/paper/2606.03188/) (2026)
- [Spatial Policy: Guiding Visuomotor Robotic Manipulation with Spatial-Aware Modeling and Reasoning](https://mlsanity.com/paper/2508.15874/) (2025)
- [DySL-VLA: Efficient Vision-Language-Action Model Inference via Dynamic-Static Layer-Skipping for Robot Manipulation](https://mlsanity.com/paper/2602.22896/) (2026)
- [Survey of Vision-Language-Action Models for Embodied Manipulation](https://mlsanity.com/paper/2508.15201/) (2025)
- [Compressor-VLA: Instruction-Guided Visual Token Compression for Efficient Robotic Manipulation](https://mlsanity.com/paper/2511.18950/) (2025)
- [TS-DP: Reinforcement Speculative Decoding For Temporal Adaptive Diffusion Policy Acceleration](https://mlsanity.com/paper/2512.15773/) (2025)
- [AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation](https://mlsanity.com/paper/2601.19634/) (2026)
- [RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics](https://mlsanity.com/paper/2603.12939/) (2026)
- [Sparse ActionGen: Accelerating Diffusion Policy with Real-time Pruning](https://mlsanity.com/paper/2601.12894/) (2026)
- [Fine-tuning Flow Matching Generative Models with Intermediate Feedback](https://mlsanity.com/paper/2510.18072/) (2025)
- [OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism](https://mlsanity.com/paper/2603.14371/) (2026)
- [Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance](https://mlsanity.com/paper/2605.24203/) (2026)
- [From Knowing to Doing Precisely: A General Self-Correction and Termination Framework for VLA models](https://mlsanity.com/paper/2602.01811/) (2026)
- [BFA++: Hierarchical Best-Feature-Aware Token Prune for Multi-View Vision Language Action Model](https://mlsanity.com/paper/2602.20566/) (2026)
- [AttenA+: Rectifying Action Inequality in Robotic Foundation Models](https://mlsanity.com/paper/2605.13548/) (2026)
- [Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference](https://mlsanity.com/paper/2607.12659/) (2026)
- [VVS: Accelerating Speculative Decoding for Visual Autoregressive Generation via Partial Verification Skipping](https://mlsanity.com/paper/2511.13587/) (2025)

## Abstract (arXiv)

Vision-Language-Action (VLA) models have attracted increasing attention for their strong control capabilities. However, their high computational cost and low execution frequency hinder their suitability for real-time tasks such as robotic manipulation and autonomous navigation. Existing VLA acceleration methods primarily focus on structural optimization, overlooking the fact that these models operate in sequential decision-making environments. As a result, temporal redundancy in sequential action generation and spatial redundancy in visual input remain unaddressed. To this end, we propose SP-VLA, a unified framework that accelerates VLA models by jointly scheduling models and pruning tokens. Specifically, we design an action-aware model scheduling mechanism that reduces temporal redundancy by dynamically switching between VLA model and a lightweight generator. Inspired by the human motion pattern of focusing on key decision points while relying on intuition for other actions, we categorize VLA actions into deliberative and intuitive, assigning the former to the VLA model and the latter to the lightweight generator, enabling frequency-adaptive execution through collaborative model scheduling. To address spatial redundancy, we further develop a spatio-semantic dual-aware token pruning method. Tokens are classified into spatial and semantic types and pruned based on their dual-aware importance to accelerate VLA inference. These two mechanisms work jointly to guide the VLA in focusing on critical actions and salient visual information, achieving effective acceleration while maintaining high accuracy. Extensive experiments show that our method achieves 1.5× lossless acceleration in LIBERO and 2.4× in SimplerEnv, with up to 6% average performance gain. Inference frequency and latency improve by 2.2× in SimplerEnv and 1.4× in LIBERO.