mlsanityData to 14 Oct 2026

Train Offline, Test Online: A Real Robot Learning Benchmark

arXiv 2306.00942 ↗First posted 1 Jun 2023cs.RO cs.AI cs.CVcited by 47

Not on the map. Why?

Cited by 21 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Three challenges limit the progress of robot learning research: robots are expensive (few labs can participate), everyone uses different robots (findings do not generalize across labs), and we lack internet-scale robotics data. We take on these challenges via a new benchmark: Train Offline, Test Online (TOTO). TOTO provides remote users with access to shared robotic hardware for evaluating methods on common tasks and an open-source dataset of these tasks for offline training. Its manipulation task suite requires challenging generalization to unseen objects, positions, and lighting. We present initial results on TOTO comparing five pretrained visual representations and four offline policy learning baselines, remotely contributed by five institutions. The real promise of TOTO, however, lies in the future: we release the benchmark for additional submissions from any user, enabling easy, direct comparison to several methods without the need to obtain hardware or collect data.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

29 of 42 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  5. 2016OpenAI Gymcited by 1,736
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  8. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  9. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  10. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  11. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  12. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
17 more
  1. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  2. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  3. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 204
  4. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  5. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 154
  6. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  7. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  8. 2016Analysis and Observations from the First Amazon Picking Challengecited by 41
  9. 2020Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulationcited by 38
  10. 2016The Robotarium: A remotely accessible swarm robotics research testbedcited by 24
  11. 2021OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulationcited by 24
  12. 2022RB2: Robotic Manipulation Benchmarking with a Twistcited by 23
  13. 2021Hierarchical Neural Dynamic Policiescited by 17
  14. 2021Benchmarking Structured Policies and Policy Optimization for Real-World Dexterous Object Manipulationcited by 15
  15. 2021Research Challenges and Progress in Robotic Grasping and Manipulation Competitionscited by 9
  16. 2019OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and researchcited by 6
  17. 2019Benchmarking Simulated Robotic Manipulation through a Real World Datasetcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 14; 2025: 14; 2026: 11.

47 papers.

Sort:

By area:

  1. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  2. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  3. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  4. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
  5. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  6. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 410
  7. 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 146
  8. 2025Unified Vision-Language-Action Modelcited by 134
  9. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 106
  10. 2024Steering Your Generalists: Improving Robotic Foundation Models via Value Guidancecited by 106
  11. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 89
  12. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
35 more
  1. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 82
  2. 2026RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robotscited by 79
  3. 2024Keypoint Action Tokens Enable In-Context Imitation Learning in Roboticscited by 74
  4. 2024Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videoscited by 69
  5. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 60
  6. 2023QUAR-VLA: Vision-Language-Action Model for Quadruped Robotscited by 56
  7. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  8. 2025AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real Worldcited by 48
  9. 2024Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learningcited by 47
  10. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 44
  11. 2023An Unbiased Look at Datasets for Visuo-Motor Pre-Trainingcited by 42
  12. 2024Diffusion Transformer Policycited by 41
  13. 2025MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulationcited by 23
  14. 2025XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representationscited by 15
  15. 2024Hearing Touch: Audio-Visual Pretraining for Contact-Rich Manipulationcited by 14
  16. 2025Reliable and Scalable Robot Policy Evaluation with Imperfect Simulatorscited by 14
  17. 2026Chain of World: World Model Thinking in Latent Motioncited by 11
  18. 2025LatBot: Distilling Universal Latent Actions for Vision-Language-Action Modelscited by 10
  19. 2025HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policiescited by 10
  20. 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learningcited by 10
  21. 2026MMaDA-VLA: Large Diffusion Vision-Language-Action Model with Unified Multi-Modal Instruction and Generationcited by 10
  22. 2023SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating Replicable Scenescited by 9
  23. 2026RotVLA: Rotational Latent Action for Vision-Language-Action Modelcited by 8
  24. 2023LHManip: A Dataset for Long-Horizon Language-Grounded Manipulation Tasks in Cluttered Tabletop Environmentscited by 7
  25. 2023Robotic Manipulation Datasets for Offline Compositional Reinforcement Learningcited by 6
  26. 2025Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLAcited by 5
  27. 2026Efficient Long-Horizon Vision-Language-Action Models via Static-Dynamic Disentanglementcited by 5
  28. 2023Sim-to-Real Causal Transfer: A Metric Learning Approach to Causally-Aware Interaction Representationscited by 4
  29. 2026AugVLA-3D: Depth-Driven Feature Augmentation for Vision-Language-Action Modelscited by 4
  30. 2024Situated Instruction Followingcited by 3
  31. 2024A Retrospective on the Robot Air Hockey Challenge: Benchmarking Robust, Reliable, and Safe Learning Techniques for Real-world Roboticscited by 1
  32. 2026TBD-VLA: Temporal Block Diffusion Vision Language Action Modelcited by 1
  33. 2025Object-Centric Action-Enhanced Representations for Robot Visuo-Motor Policy Learningcited by 0
  34. 2026Critical Interval MSE: Toward Reliable Offline Validation for Robot Manipulation Policiescited by 0
  35. 2026UniJEPA: A Unified Joint-Embedding Predictive Architecture for Task-Agnostic Visual World Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Monkey See, Can Monkey Do? A Benchmark for Evaluating Robot Skill Learning by Observation
  2. 2023Benchmarking Offline Reinforcement Learning on Real-Robot Hardware
  3. 2024MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learning
  4. 2019RLBench: The Robot Learning Benchmark & Learning Environment
  5. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials
  6. 2026ArmnetBench v0.1: Parallel Real-World Evaluation of Manipulation Policies on a Low-Cost Arm Farm
  7. 2024Octo: An Open-Source Generalist Robot Policy
  8. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulation