mlsanityData to 7 Oct 2026

HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model

arXiv 2503.10631 ↗First posted 13 Mar 2025cs.CV cs.ROcited by 176

On the map: Vision-language-action models → Efficient VLA inference and adaptation

Abstract (arXiv)

A fundamental objective of manipulation policy design is to endow robots to comprehend human instructions, reason about scene cues, and execute generalized actions in dynamic environments. Recent autoregressive vision-language-action (VLA) methods inherit common-sense reasoning capabilities from vision-language models (VLMs) for next action-token prediction. However, these methods quantize actions into discrete bins, which disrupts the continuity required for precise control. In contrast, existing diffusion-based VLA methods incorporate an additional diffusion head to predict continuous actions solely conditioned on feature representations extracted by the VLM, without fully leveraging the VLM's pretrained reasoning capabilities through token-level generation. To address these limitations, we introduce HybridVLA, a unified framework that absorbs the continuous nature of diffusion-based actions and the contextual reasoning of autoregression within a single large language model. To mitigate interference between the two generation paradigms, we propose a collaborative training recipe that seamlessly incorporates diffusion denoising into the next-token prediction process. With this recipe, we find these two action prediction methods not only reinforce each other but also exhibit varying strength across different tasks. Therefore, we design a collaborative action ensemble mechanism that adaptively fuses both predictions, leading to more robust control. HybridVLA outperforms previous state-of-the-art VLA methods by 14% and 19% in mean success rate on simulation and real-world tasks, respectively, while demonstrating stable manipulation in unseen configurations.

Key facts. Cited by 176 papers in the map (data through 7 Oct 2026). Placed in Efficient VLA inference and adaptation, one of 13 directions in Vision-language-action models. Builds on 101 of its 116 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

101 of 116 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Models16 Jan

Other references · 100

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
38 more
  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  7. 2022Video Diffusion Modelscited by 2,280
  8. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  11. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  12. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  13. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  14. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  15. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  16. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  17. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  18. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  19. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  20. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  21. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  22. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  23. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  24. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  25. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  26. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  27. 2018Behavioral Cloning from Observationcited by 482
  28. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  29. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  30. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  31. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  32. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  33. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  34. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  35. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  36. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  37. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  38. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331

50 most cited of 100 All 100 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 82; 2026: 94.

176 papers.

Sort:

By area:

  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  2. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  3. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  4. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 183
  5. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  6. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  7. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  8. 2025Unified Vision-Language-Action Modelcited by 128
  9. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  10. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  11. 2025Galaxea Open-World Dataset and G0 Dual-System VLA Modelcited by 90
  12. 2025ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulationcited by 87
38 more
  1. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  2. 2025CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsificationcited by 70
  3. 2025GeoVLA: Empowering 3D Representations in Vision-Language-Action Modelscited by 68
  4. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  5. 2025WorldEval: World Model as Real-World Robot Policies Evaluatorcited by 64
  6. 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understandingcited by 55
  7. 2025Training Strategies for Efficient Embodied Reasoningcited by 53
  8. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  9. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  10. 2026Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Executioncited by 42
  11. 2026World Model for Robot Learning: A Comprehensive Surveycited by 42
  12. 2025H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulationcited by 41
  13. 2026AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routingcited by 41
  14. 2026VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Modelcited by 39
  15. 2025CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learningcited by 37
  16. 2025Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignmentcited by 35
  17. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  18. 2025EO-1: An Open Unified Embodied Foundation Model for General Robot Controlcited by 30
  19. 2025TA-VLA: Elucidating the Design Space of Torque-aware Vision-Language-Action Modelscited by 30
  20. 2025dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thoughtcited by 30
  21. 2026Native Video-Action Pretraining for Generalizable Robot Controlcited by 30
  22. 2025RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaborationcited by 28
  23. 2025Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoningcited by 28
  24. 2026LaST0: Latent Spatio-Temporal Chain-of-Thought for Robotic Vision-Language-Action Modelcited by 28
  25. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  26. 2026When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAscited by 27
  27. 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Braincited by 24
  28. 2026Adaptive Action Chunking at Inference-time for Vision-Language-Action Modelscited by 24
  29. 2025AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulationcited by 23
  30. 2025MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulationcited by 22
  31. 2025AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulationcited by 21
  32. 2025Enhancing Generalization in Vision-Language-Action Models by Preserving Pretrained Representationscited by 21
  33. 2025SemanticVLA: Semantic-Aligned Sparsification and Enhancement for Efficient Robotic Manipulationcited by 20
  34. 2026A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulationcited by 20
  35. 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Modelscited by 19
  36. 2026Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulationcited by 19
  37. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  38. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 17

50 most cited of 176 All 176 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025LLaDA-VLA: Vision Language Diffusion Action Models
  2. 2026AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models
  3. 2026MMaDA-VLA: Large Diffusion Vision-Language-Action Model with Unified Multi-Modal Instruction and Generation
  4. 2026dVLA-RL: Reinforcement Learning over Denoising Trajectories for Discrete Diffusion Vision-Language-Action Models
  5. 2026Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference
  6. 2026G0.5: One Autoregressive Stream for Robot Reasoning and Action
  7. 2026A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model
  8. 2026Catch Me If You Can: Real-Time Feedback Denoising for Responsive VLAs