BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuning
On the map: LLM reasoning → Spatial and embodied reasoning
also in 2 more
Abstract (arXiv)
While autoregressive (AR) Vision-Language-Action (VLA) models have demonstrated formidable reasoning capabilities in robotic tasks, their sequential decoding process often incurs high inference latency and may amplify error accumulation during long-horizon execution. Discrete Diffusion Language Models (dLLMs) provide a promising alternative through parallel token refinement, but their practical deployment in robotics remains limited by repeated denoising function evaluations (NFEs) and the difficulty of directly applying standard KV caching to bidirectional iterative decoding. To bridge these paradigms, we propose BlockVLA, a framework that adapts pretrained AR backbones into an efficient discrete diffusion policy through a block diffusion paradigm. BlockVLA maintains autoregressive dependencies at the block level while enabling parallel denoising within each block, thereby combining global causal coherence with local parallel generation. This design enables prefix KV-cache reuse across completed blocks, reduces the effective cost of iterative denoising, and provides a smoother transition from AR pretraining to diffusion-based policy fine-tuning. We conduct extensive evaluations on the LIBERO and SimplerEnv benchmarks. Experimental results demonstrate that our BlockVLA achieves a 3.3× inference acceleration over standard discrete diffusion baselines. Furthermore, our model exhibits superior training efficiency, with success rates converging substantially faster than baselines, a gain that is particularly pronounced in complex, long-horizon tasks, where BlockVLA achieves significant performance gains in the early stages of training. This work establishes Block Diffusion as a robust bridge between large-scale pretrained AR models and efficient, high-frequency real-time robotic control.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 57 of its 58 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
57 of 58 references.
Closest ancestors · 1
Same direction, newest first
Other references · 56
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2023Qwen Technical Reportcited by 3,273
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
38 more
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2025Large Language Diffusion Modelscited by 872
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
- 2024PaliGemma: A versatile 3B VLM for transfercited by 679
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
- 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
- 2025Dream 7B: Diffusion Large Language Modelscited by 515
- 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
- 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
- 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
- 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
- 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
- 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
- 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
- 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
- 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
- 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
- 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
- 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
- 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
- 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
- 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
- 2025OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulationcited by 79
- 2025VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizerscited by 67
- 2025VLAS: Vision-Language-Action Model With Speech Instructions For Customized Robot Manipulationcited by 48
- 2025Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Processcited by 37
- 2025LLaDA-VLA: Vision Language Diffusion Action Modelscited by 30
- 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 30
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM
- 2026BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation
- 2026Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving
- 2026dVLA-RL: Reinforcement Learning over Denoising Trajectories for Discrete Diffusion Vision-Language-Action Models
- 2026Toward Real-Time VLAs: Stage-Aware Two-Step Flow Denoising and System-Level Evaluation
- 2025dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thought
- 2026MMaDA-VLA: Large Diffusion Vision-Language-Action Model with Unified Multi-Modal Instruction and Generation
- 2025FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenization