mlsanityData to 7 Oct 2026

π0: A Vision-Language-Action Flow Model for General Robot Control

arXiv 2410.24164 ↗First posted 31 Oct 2024cs.LG cs.ROcited by 2,297

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

Robot learning holds tremendous promise to unlock the full potential of flexible, general, and dexterous robot systems, as well as to address some of the deepest questions in artificial intelligence. However, bringing robot learning to the level of generality required for effective real-world systems faces major obstacles in terms of data, generalization, and robustness. In this paper, we discuss how generalist robot policies (i.e., robot foundation models) can address these challenges, and how we can design effective generalist robot policies for complex and highly dexterous tasks. We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We then discuss how this model can be trained on a large and diverse dataset from multiple dexterous robot platforms, including single-arm robots, dual-arm robots, and mobile manipulators. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people and from a high-level VLM policy, and its ability to acquire new skills via fine-tuning. Our results cover a wide variety of tasks, such as laundry folding, table cleaning, and assembling boxes.

Key facts. Cited by 2,297 papers in the map (data through 7 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 52 of its 62 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

52 of 62 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation19 Sep
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Model13 Jun
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2022Flow Matching for Generative Modelingcited by 5,426
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
37 more
  1. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  9. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  10. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  12. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  13. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  14. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  15. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  16. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  17. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  18. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  19. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  20. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  21. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  22. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  23. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  24. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  25. 2022Rectified Flow: A Marginal Preserving Approach to Optimal Transportcited by 277
  26. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  27. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  28. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  29. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  30. 2024ALOHA Unleashed: A Simple Recipe for Robot Dexteritycited by 128
  31. 2024Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Modelscited by 113
  32. 2023On Bringing Robots Homecited by 101
  33. 2018Robot Learning in Homes: Improving Generalization and Reducing Dataset Biascited by 75
  34. 2024ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationcited by 61
  35. 2024MARS: Mixture of Auto-Regressive Models for Fine-grained Text-to-image Synthesiscited by 44
  36. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  37. 2016More than a Million Ways to Be Pushed: A High-Fidelity Experimental Dataset of Planar Pushingcited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 742; 2026: 1544.

2,297 papers.

Sort:

By area:

  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  2. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  3. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  4. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  5. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  6. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  7. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  8. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  9. 2026World Action Models are Zero-shot Policiescited by 411
  10. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  11. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  12. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
38 more
  1. 2026Causal World Modeling for Robot Controlcited by 289
  2. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  3. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  4. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
  5. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  6. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
  7. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  8. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  9. 2025Unified Video Action Modelcited by 195
  10. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  11. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  12. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 183
  13. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
  14. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  15. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  16. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
  17. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
  18. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  19. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  20. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 166
  21. 2025Flow Q-Learningcited by 162
  22. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters Morecited by 149
  23. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  24. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  25. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  26. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  27. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
  28. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
  29. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  30. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
  31. 2025Interactive Post-Training for Vision-Language-Action Modelscited by 129
  32. 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Modelcited by 129
  33. 2025Unified Vision-Language-Action Modelcited by 128
  34. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
  35. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  36. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  37. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  38. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125

50 most cited of 2,297 All 2,297 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models
  2. 2023Programmatically Grounded, Compositionally Generalizable Robotic Manipulation
  3. 2026Improving Robotic Generalist Policies via Flow Reversal Steering
  4. 2023Transferring Foundation Models for Generalizable Robotic Manipulation
  5. 2026MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation
  6. 2026Task-Prototype Guided Flow Matching for Few-Shot Generalization in Vision-Language Robot Manipulation
  7. 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models
  8. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges