mlsanityData to 7 Oct 2026

Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance

arXiv 2507.22424 ↗First posted 30 Jul 2025cs.LG cs.AIcited by 31

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 2 more
Abstract (arXiv)

Vision-Language-Action (VLA) models have made substantial progress by leveraging the robust capabilities of Visual Language Models (VLMs). However, VLMs' significant parameter size and autoregressive (AR) decoding nature impose considerable computational demands on VLA models. While Speculative Decoding (SD) has shown efficacy in accelerating Large Language Models (LLMs) by incorporating efficient drafting and parallel verification, allowing multiple tokens to be generated in one forward pass, its application to VLA models remains unexplored. This work introduces Spec-VLA, an SD framework designed to accelerate VLA models. Due to the difficulty of the action prediction task and the greedy decoding mechanism of the VLA models, the direct application of the advanced SD framework to the VLA prediction task yields a minor speed improvement. To boost the generation speed, we propose an effective mechanism to relax acceptance utilizing the relative distances represented by the action tokens of the VLA model. Empirical results across diverse test scenarios affirm the effectiveness of the Spec-VLA framework, and further analysis substantiates the impact of our proposed strategies, which enhance the acceptance length by 44%, achieving 1.42 times speedup compared with the OpenVLA baseline, without compromising the success rate. The success of the Spec-VLA framework highlights the potential for broader application of speculative execution in VLA prediction scenarios.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 38 references that are in the map. First posted on arXiv 30 Jul 2025.

Builds on

30 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  2. 2024MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric12 Mar

Other references · 28

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  5. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  6. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  7. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  8. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  9. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  10. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  11. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  12. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 318
16 more
  1. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  2. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
  3. 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 116
  4. 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 102
  5. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 102
  6. 2025PD-VLA: Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decodingcited by 88
  7. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  8. 2024Learning Harmonized Representations for Speculative Samplingcited by 69
  9. 2024Kangaroo: Lossless Self-Speculative Decoding via Double Early Exitingcited by 58
  10. 2024LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decodingcited by 46
  11. 2024RoboNurse-VLA: Robotic Scrub Nurse System based on Vision-Language-Action Modelcited by 28
  12. 2024Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Controlcited by 25
  13. 2024RAT: Adversarial Attacks on Deep Reinforcement Agents for Targeted Behaviorscited by 16
  14. 2024Speculative Decoding with CTC-based Draft Model for LLM Inference Accelerationcited by 14
  15. 2025LANTERN++: Enhancing Relaxed Speculative Decoding with Static Tree Drafting for Visual Auto-regressive Modelscited by 11
  16. 2024Efficient Preference-based Reinforcement Learning via Aligned Experience Estimationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 24.

31 papers.

Sort:

By area:

  1. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  2. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  3. 2025Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Surveycited by 25
  4. 2026FASTER: Rethinking Real-Time Flow VLAscited by 20
  5. 2026DyQ-VLA: Temporal-Dynamic-Aware Quantization for Embodied Vision-Language-Action Modelscited by 10
  6. 2026KERV: Kinematic-Rectified Speculative Decoding for Embodied VLA Modelscited by 9
  7. 2026DySL-VLA: Efficient Vision-Language-Action Model Inference via Dynamic-Static Layer-Skipping for Robot Manipulationcited by 8
  8. 2026Realtime-VLA FLASH: Speculative Inference Framework for Diffusion-based VLAscited by 8
  9. 2025ActionFlow: A Pipelined Action Acceleration for Vision Language Models on Edgecited by 6
  10. 2026HeiSD: Hybrid Speculative Decoding for Embodied Vision-Language-Action Models with Kinematic Awarenesscited by 6
  11. 2026Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deploymentcited by 6
  12. 2025Experiences from Benchmarking Vision-Language-Action Models for Robotic Manipulationcited by 5
19 more
  1. 2026TIDAL: Temporally Interleaved Diffusion and Action Loop for High-Frequency VLA Controlcited by 5
  2. 2026Open-Loop Planning, Closed-Loop Verification: Speculative Verification for VLAcited by 5
  3. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  4. 2026Action Draft and Verify: A Self-Verifying Framework for Vision-Language-Action Modelcited by 4
  5. 2026Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulationcited by 4
  6. 2025When, What, and How: Rethinking Retrieval-Enhanced Speculative Decodingcited by 3
  7. 2026ProbeFlow: Training-Free Adaptive Flow Matching for Vision-Language-Action Modelscited by 3
  8. 2026Dynamic Execution Commitment of Vision-Language-Action Modelscited by 2
  9. 2026Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Modelscited by 2
  10. 2026PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Modelscited by 2
  11. 2026CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Interventioncited by 2
  12. 2026Habilis-β: A Fast-Motion and Long-Lasting On-Device Vision-Language-Action Modelcited by 1
  13. 2026Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Thinkcited by 1
  14. 2026Draft-and-Target Sampling for Video Generation Policycited by 0
  15. 2026VLA-InfoEntropy: A Training-Free Vision-Attention Information Entropy Approach for Vision-Language-Action Models Inference Acceleration and Successcited by 0
  16. 2026Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inferencecited by 0
  17. 2026WA-SpecDec: World-Aware Speculative Decoding for Vision-Language-Action Modelscited by 0
  18. 2026Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verificationcited by 0
  19. 2026Is Multimodal Speculative Decoding Ready for Diffusion-Based Parallel Drafting? A Survey and Empirical Diagnosiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SpecVLM: Fast Speculative Decoding in Vision-Language Models
  2. 2025Spec-LLaVA: Accelerating Vision-Language Models with Dynamic Tree-Based Speculative Decoding
  3. 2026MMSpec: Benchmarking Speculative Decoding for Vision-Language Models
  4. 2025FastVLM: Self-Speculative Decoding for Fast Vision-Language Model Inference
  5. 2026DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation
  6. 2025DREAM: Drafting with Refined Target Features and Entropy-Adaptive Cross-Attention Fusion for Multimodal Speculative Decoding
  7. 2025MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models
  8. 2025HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models