mlsanityData to 7 Oct 2026

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

arXiv 2310.08864 ↗First posted 13 Oct 2023cs.ROcited by 849

Not on the map. Why?

Cited by 320 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website https://robotics-transformer-x.github.io.

Key facts. Cited by 849 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Oct 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 203; 2025: 297; 2026: 332.

849 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  2. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  3. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  4. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  5. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  6. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  7. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  8. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  9. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  10. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  11. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  12. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
38 more
  1. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  2. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  3. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  4. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  5. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  6. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  7. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  8. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  9. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  10. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  11. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  12. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  13. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  14. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 183
  15. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  16. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  17. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 167
  18. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  19. 2025Flow Q-Learningcited by 162
  20. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  21. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 152
  22. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 146
  23. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  24. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  25. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
  26. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  27. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  28. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 134
  29. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 133
  30. 2023Large Language Models for Robotics: A Surveycited by 132
  31. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  32. 2025Unified Vision-Language-Action Modelcited by 128
  33. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126
  34. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  35. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  36. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
  37. 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 116
  38. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116

50 most cited of 849 All 849 (JSON)