mlsanityData to 7 Oct 2026

OpenVLA: An Open-Source Vision-Language-Action Model

arXiv 2406.09246 ↗First posted 13 Jun 2024cs.RO cs.LGcited by 2,777

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

Large policies pretrained on a combination of Internet-scale vision-language data and diverse robot demonstrations have the potential to change how we teach robots new skills: rather than training new behaviors from scratch, we can fine-tune such vision-language-action (VLA) models to obtain robust, generalizable policies for visuomotor control. Yet, widespread adoption of VLAs for robotics has been challenging as 1) existing VLAs are largely closed and inaccessible to the public, and 2) prior work fails to explore methods for efficiently fine-tuning VLAs for new tasks, a key component for adoption. Addressing these challenges, we introduce OpenVLA, a 7B-parameter open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA builds on a Llama 2 language model combined with a visual encoder that fuses pretrained features from DINOv2 and SigLIP. As a product of the added data diversity and new model components, OpenVLA demonstrates strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate across 29 tasks and multiple robot embodiments, with 7x fewer parameters. We further show that we can effectively fine-tune OpenVLA for new settings, with especially strong generalization results in multi-task environments involving multiple objects and strong language grounding abilities, and outperform expressive from-scratch imitation learning methods such as Diffusion Policy by 20.4%. We also explore compute efficiency; as a separate contribution, we show that OpenVLA can be fine-tuned on consumer GPUs via modern low-rank adaptation methods and served efficiently via quantization without a hit to downstream success rate. Finally, we release model checkpoints, fine-tuning notebooks, and our PyTorch codebase with built-in support for training VLAs at scale on Open X-Embodiment datasets.

Key facts. Cited by 2,777 papers in the map (data through 7 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 82 of its 110 references that are in the map. First posted on arXiv 13 Jun 2024.

Builds on

82 of 110 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 81

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  11. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  12. 2023Qwen Technical Reportcited by 3,273
38 more
  1. 2023Mistral 7Bcited by 3,065
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2019Towards VQA Models That Can Readcited by 1,696
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  13. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  14. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  15. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  16. 2016Modeling Context in Referring Expressionscited by 1,054
  17. 2022A Generalist Agentcited by 802
  18. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  19. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  20. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  21. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  22. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  23. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  24. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  25. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  26. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  27. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 398
  28. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  29. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  30. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 336
  31. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
  32. 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
  33. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  34. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  35. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 230
  36. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  37. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  38. 2023ViNT: A Foundation Model for Visual Navigationcited by 210

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 137; 2025: 914; 2026: 1726.

2,777 papers.

Sort:

By area:

  1. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  2. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  3. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  4. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  5. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  6. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  7. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  8. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  9. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  10. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  11. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  12. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
38 more
  1. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  2. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  3. 2026World Action Models are Zero-shot Policiescited by 411
  4. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  5. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  6. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  7. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
  8. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  9. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
  10. 2024Latent Action Pretraining from Videoscited by 302
  11. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 298
  12. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  13. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  14. 2026Causal World Modeling for Robot Controlcited by 289
  15. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  16. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
  17. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  18. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  19. 2025Motus: A Unified Latent Action World Modelcited by 231
  20. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  21. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  22. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 216
  23. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  24. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  25. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  26. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  27. 2025Unified Video Action Modelcited by 195
  28. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  29. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  30. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  31. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 183
  32. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  33. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
  34. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
  35. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
  36. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  37. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  38. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 166

50 most cited of 2,777 All 2,777 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Review
  2. 2026FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies
  3. 2025EdgeVLA: Efficient Vision-Language-Action Models
  4. 2026Offline Semantic Guidance for Efficient Vision-Language-Action Policy Distillation
  5. 2026Two Bridges, One Pathway: From VLMs to Generalizable VLAs with Embodied Trajectory-Coupled Data
  6. 2026A Pragmatic VLA Foundation Model
  7. 2025Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion
  8. 2026Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering