mlsanityData to 7 Oct 2026

Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation

arXiv 2408.11812 ↗First posted 21 Aug 2024cs.RO cs.LGcited by 108

Not on the map. Why?

Cited by 47 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Modern machine learning systems rely on large datasets to attain broad generalization, and this often poses a challenge in robot learning, where each robotic platform and task might have only a small dataset. By training a single policy across many different kinds of robots, a robot learning method can leverage much broader and more diverse datasets, which in turn can lead to better generalization and robustness. However, training a single policy on multi-robot data is challenging because robots can have widely varying sensors, actuators, and control frequencies. We propose CrossFormer, a scalable and flexible transformer-based policy that can consume data from any embodiment. We train CrossFormer on the largest and most diverse dataset to date, 900K trajectories across 20 different robot embodiments. We demonstrate that the same network weights can control vastly different robots, including single and dual arm manipulation systems, wheeled robots, quadcopters, and quadrupeds. Unlike prior work, our model does not require manual alignment of the observation or action spaces. Extensive experiments in the real world show that our method matches the performance of specialist policies tailored for each embodiment, while also significantly outperforming the prior state of the art in cross-embodiment learning.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 58 of its 71 references that are in the map. First posted on arXiv 21 Aug 2024.

Builds on

58 of 71 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  7. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  8. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  9. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  10. 2017Hindsight Experience Replaycited by 1,096
  11. 2021Scaling Vision Transformerscited by 993
  12. 2022A Generalist Agentcited by 802
38 more
  1. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  2. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  3. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  5. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  6. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  7. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  8. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  9. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  10. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  11. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  12. 2023ViNT: A Foundation Model for Visual Navigationcited by 210
  13. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  14. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  15. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  16. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  17. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  18. 2022GNM: A General Navigation Model to Drive Any Robotcited by 161
  19. 2023Structured World Models from Human Videoscited by 144
  20. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
  21. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  22. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
  23. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 129
  24. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  25. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 114
  26. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  27. 2022Grounding Language with Visual Affordances over Unstructured Datacited by 101
  28. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  29. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulationcited by 82
  30. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  31. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  32. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
  33. 2018Robot Learning in Homes: Improving Generalization and Reducing Dataset Biascited by 75
  34. 2023MUTEX: Learning Unified Policies from Multimodal Task Specificationscited by 70
  35. 2024Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigationcited by 60
  36. 2023Multi-Stage Cable Routing through Hierarchical Imitation Learningcited by 59
  37. 2023HYDRA: Hybrid Robot Actions for Imitation Learningcited by 54
  38. 2018Jointly Learning to Construct and Control Agents using Deep Reinforcement Learningcited by 51

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 48; 2026: 43.

108 papers.

Sort:

By area:

  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  2. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  3. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  4. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  5. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  6. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  7. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  8. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
  9. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  10. 2025GR-3 Technical Reportcited by 110
  11. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
  12. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
38 more
  1. 2025Universal Actions for Enhanced Embodied Foundation Modelscited by 80
  2. 2024The Ingredients for Robotic Diffusion Transformerscited by 72
  3. 2024Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videoscited by 65
  4. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 63
  5. 2025FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policiescited by 56
  6. 2026MEM: Multi-Scale Embodied Memory for Vision Language Action Modelscited by 55
  7. 2025Training Strategies for Efficient Embodied Reasoningcited by 53
  8. 2025AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real Worldcited by 46
  9. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  10. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  11. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Modelcited by 43
  12. 2024From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessonscited by 42
  13. 2026World Model for Robot Learning: A Comprehensive Surveycited by 42
  14. 2025GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulationcited by 41
  15. 2025Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Groundingcited by 40
  16. 2025CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrationscited by 39
  17. 2025Emergence of Human to Robot Transfer in Vision-Language-Action Modelscited by 36
  18. 2025OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigationcited by 34
  19. 2025Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentationcited by 31
  20. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  21. 2024Cross-Embodiment Dexterous Grasping with Reinforcement Learningcited by 24
  22. 2025Embodied AI: From LLMs to World Modelscited by 23
  23. 2026GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learningcited by 23
  24. 2025Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimizationcited by 20
  25. 2026A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulationcited by 20
  26. 2025Refined Policy Distillation: From VLA Generalists to RL Expertscited by 19
  27. 2025Is Diversity All You Need for Scalable Robotic Manipulation?cited by 19
  28. 2024Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitationcited by 18
  29. 2026LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfercited by 17
  30. 2026Show-Harness: Just a VLM Agent Can Play Robotscited by 17
  31. 2025CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-Worldcited by 16
  32. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 15
  33. 2025RobotArena ∞: Scalable Robot Benchmarking via Real-to-Sim Translationcited by 15
  34. 2025A Step Toward World Models: A Survey on Robotic Manipulationcited by 15
  35. 2024GHIL-Glue: Hierarchical Control with Filtered Subgoal Imagescited by 14
  36. 2025LocoFormer: Generalist Locomotion via Long-context Adaptationcited by 14
  37. 2024STEER: Flexible Robotic Manipulation via Dense Language Groundingcited by 13
  38. 2024SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuumcited by 13

50 most cited of 108 All 108 (JSON)