mlsanityData to 7 Oct 2026

Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation

arXiv 1810.07121 ↗First posted 16 Oct 2018cs.RO cs.CV cs.LGcited by 76

Not on the map. Why?

Cited by 31 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In recent years, we have seen an emergence of data-driven approaches in robotics. However, most existing efforts and datasets are either in simulation or focus on a single task in isolation such as grasping, pushing or poking. In order to make progress and capture the space of manipulation, we would need to collect a large-scale dataset of diverse tasks such as pouring, opening bottles, stacking objects etc. But how does one collect such a dataset? In this paper, we present the largest available robotic-demonstration dataset (MIME) that contains 8260 human-robot demonstrations over 20 different robotic tasks (https://sites.google.com/view/mimedataset). These tasks range from the simple task of pushing objects to the difficult task of stacking household objects. Our dataset consists of videos of human demonstrations and kinesthetic trajectories of robot demonstrations. We also propose to use this dataset for the task of mapping 3rd person video features to robot trajectories. Furthermore, we present two different approaches using this dataset and evaluate the predicted robot trajectories against ground-truth trajectories. We hope our dataset inspires research in multiple areas including visual imitation, trajectory prediction, and multi-task robotic learning.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 2; 2021: 12; 2022: 14; 2023: 10; 2024: 18; 2025: 16; 2026: 3.

76 papers.

Sort:

By area:

  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  2. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  3. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  4. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  5. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  6. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  7. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  8. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  9. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  10. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  11. 2022Human-to-Robot Imitation in the Wildcited by 157
  12. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
38 more
  1. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
  2. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  3. 2023On Bringing Robots Homecited by 101
  4. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  5. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 90
  6. 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 83
  7. 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agentscited by 83
  8. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  9. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 79
  10. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 78
  11. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 75
  12. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  13. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controllercited by 73
  14. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68
  15. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  16. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  17. 2022Imitation Learning by Estimating Expertise of Demonstratorscited by 50
  18. 2023Zero-Shot Robot Manipulation from Passive Human Videoscited by 49
  19. 2023Train Offline, Test Online: A Real Robot Learning Benchmarkcited by 47
  20. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  21. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  22. 2024Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learningcited by 44
  23. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 43
  24. 2024Efficient Data Collection for Robotic Manipulation via Compositional Generalizationcited by 35
  25. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  26. 2021RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learningcited by 31
  27. 2023Polybot: Training One Policy Across Robots While Embracing Variabilitycited by 28
  28. 2025DataMIL: Selecting Data for Robot Imitation Learning with Datamodelscited by 25
  29. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  30. 2025Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulationcited by 22
  31. 2024Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusioncited by 21
  32. 2024RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agentscited by 20
  33. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  34. 2025Robot Learning from a Physical World Modelcited by 18
  35. 2021Shaping embodied agent behavior with activity-context priors from egocentric videocited by 17
  36. 2022Cross-Domain Transfer via Semantic Skill Imitationcited by 17
  37. 2022Dungeons and Data: A Large-Scale NetHack Datasetcited by 16
  38. 2021Playful Interactions for Representation Learningcited by 13

50 most cited of 76 All 76 (JSON)