mlsanityData to 7 Oct 2026

RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot

arXiv 2307.00595 ↗First posted 2 Jul 2023cs.RO cs.AI cs.CVcited by 173

Not on the map. Why?

Cited by 69 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This feature is attractive for enabling robots to acquire new skills and improving task and motion planning. However, due to limitations in the training dataset, the current focus of the community has mainly been on simple cases, such as push or pick-place tasks, relying solely on visual guidance. In reality, there are many complex skills, some of which may even require both visual and tactile perception to solve. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception. To achieve this, we have collected a dataset comprising over 110,000 contact-rich robot manipulation sequences across diverse skills, contexts, robots, and camera viewpoints, all collected in the real world. Each sequence in the dataset includes visual, force, audio, and action information. Moreover, we also provide a corresponding human demonstration video and a language description for each robot sequence. We have invested significant efforts in calibrating all the sensors and ensuring a high-quality dataset. The dataset is made publicly available at rh20t.github.io

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 2 Jul 2023.

Builds on

30 of 46 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Segment Anythingcited by 8,158
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  8. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  9. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  10. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  11. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  12. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
18 more
  1. 2017One-Shot Imitation Learningcited by 232
  2. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  3. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  4. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  5. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  6. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  7. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  8. 2018Zero-Shot Visual Imitationcited by 114
  9. 2020Visual Imitation Made Easycited by 111
  10. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  11. 2019Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Taskscited by 80
  12. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  13. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 65
  14. 2018Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstrationcited by 60
  15. 2021Towards More Generalizable One-shot Visual Imitation Learningcited by 48
  16. 2019Learning One-Shot Imitation from Humans without Humanscited by 32
  17. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Rewardcited by 26
  18. 2020Learning Dense Rewards for Contact-Rich Manipulation Taskscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 17; 2025: 75; 2026: 78.

173 papers.

Sort:

By area:

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  4. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  5. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  6. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  7. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  8. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  9. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  10. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  11. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  12. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters Morecited by 149
38 more
  1. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  2. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  3. 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 134
  4. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
  5. 2025Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulationcited by 103
  6. 2025PointVLA: Injecting the 3D World into Vision-Language-Action Modelscited by 101
  7. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  8. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  9. 2025EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Modelscited by 89
  10. 2025PD-VLA: Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decodingcited by 88
  11. 2025ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulationcited by 87
  12. 2025VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Cachingcited by 83
  13. 2025Universal Actions for Enhanced Embodied Foundation Modelscited by 80
  14. 2025ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceivercited by 80
  15. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 68
  16. 2025VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizerscited by 67
  17. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  18. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 66
  19. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 63
  20. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  21. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 61
  22. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 59
  23. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  24. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 49
  25. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 47
  26. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 47
  27. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  28. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  29. 2026MolmoAct2: Action Reasoning Models for Real-world Deploymentcited by 45
  30. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  31. 2025H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulationcited by 41
  32. 2025CEED-VLA: Consistency Vision-Language-Action Model with Early-Exit Decodingcited by 38
  33. 2025BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimizationcited by 37
  34. 2025CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learningcited by 37
  35. 2024Assessment of Multimodal Large Language Models in Alignment with Human Valuescited by 36
  36. 2024ForceMimic: Force-Centric Imitation Learning with Force-Motion Capture System for Contact-Rich Manipulationcited by 36
  37. 2025DeepVerse: 4D Autoregressive Video Generation as a World Modelcited by 36
  38. 20253DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Modelcited by 36

50 most cited of 173 All 173 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
  2. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos
  3. 2024RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents
  4. 2025See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations
  5. 2026Human-to-Robot Interaction: Learning from Video Demonstration for Robot Imitation
  6. 2025RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments
  7. 2026Simple-to-Complex Structured Demonstrations for Vision-Language-Action Learning
  8. 2022VIMA: General Robot Manipulation with Multimodal Prompts