mlsanityData to 7 Oct 2026

RoboNet: Large-Scale Multi-Robot Learning

arXiv 1910.11215 ↗First posted 24 Oct 2019cs.RO cs.CV cs.LGcited by 268

Not on the map. Why?

Cited by 49 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Robot learning has emerged as a promising tool for taming the complexity and diversity of the real world. Methods based on high-capacity models, such as deep networks, hold the promise of providing effective generalization to a wide range of open-world environments. However, these same methods typically require large amounts of diverse training data to generalize effectively. In contrast, most robotic learning experiments are small-scale, single-domain, and single-robot. This leads to a frequent tension in robotic learning: how can we learn generalizable robotic controllers without having to collect impractically large amounts of data for each separate experiment? In this paper, we propose RoboNet, an open database for sharing robotic experience, which provides an initial pool of 15 million video frames, from 7 different robot platforms, and study how it can be used to learn generalizable models for vision-based robotic manipulation. We combine the dataset with two different learning algorithms: visual foresight, which uses forward video prediction models, and supervised inverse models. Our experiments test the learned algorithms' ability to work across new objects, new tasks, new scenes, new camera viewpoints, new grippers, or even entirely new robots. In our final experiment, we find that by pre-training on RoboNet and fine-tuning on data from a held-out Franka or Kuka robot, we can exceed the performance of a robot-specific training approach that uses 4x-20x more data. For videos and data, see the project webpage: https://www.robonet.wiki/

Key facts. Cited by 268 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 4; 2021: 21; 2022: 37; 2023: 44; 2024: 45; 2025: 56; 2026: 61.

268 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  6. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  7. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  8. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  9. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  10. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  11. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  12. 2023Learning Interactive Real-World Simulatorscited by 440
38 more
  1. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  2. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  3. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  4. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  5. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  6. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  7. 2023ViNT: A Foundation Model for Visual Navigationcited by 210
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  9. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  10. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  11. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  12. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  13. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  14. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  15. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  16. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  17. 2022GNM: A General Navigation Model to Drive Any Robotcited by 161
  18. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  19. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 159
  20. 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 155
  21. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulationcited by 149
  22. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 145
  23. 2023Structured World Models from Human Videoscited by 144
  24. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  25. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  26. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
  27. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 134
  28. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
  29. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  30. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  31. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  32. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  33. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
  34. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  35. 2025GR-3 Technical Reportcited by 110
  36. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
  37. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulationcited by 108
  38. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108

50 most cited of 268 All 268 (JSON)