mlsanityData to 7 Oct 2026

Simitate: A Hybrid Imitation Learning Benchmark

arXiv 1905.06002 ↗First posted 15 May 2019cs.LG cs.RO stat.MLcited by 10

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

We present Simitate --- a hybrid benchmarking suite targeting the evaluation of approaches for imitation learning. A dataset containing 1938 sequences where humans perform daily activities in a realistic environment is presented. The dataset is strongly coupled with an integration into a simulator. RGB and depth streams with a resolution of 960mathbb×540 at 30Hz and accurate ground truth poses for the demonstrator's hand, as well as the object in 6 DOF at 120Hz are provided. Along with our dataset we provide the 3D model of the used environment, labeled object images and pre-trained models. A benchmarking suite that aims at fostering comparability and reproducibility supports the development of imitation learning approaches. Further, we propose and integrate evaluation metrics on assessing the quality of effect and trajectory of the imitation performed in simulation. Simitate is available on our project website: urlhttps://agas.uni-koblenz.de/data/simitate/.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 51 references that are in the map. First posted on arXiv 15 May 2019.

Builds on

26 of 51 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2017Mask R-CNNcited by 7,259
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2014Going Deeper with Convolutionscited by 4,231
  6. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  9. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  10. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 531
  11. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  12. 2018An Algorithmic Perspective on Imitation Learningcited by 455
14 more
  1. 2016Convolutional Pose Machinescited by 325
  2. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  3. 2017One-Shot Imitation Learningcited by 232
  4. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  5. 2015Visual Semantic Role Labelingcited by 188
  6. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  7. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  8. 2017Deep Extreme Cut: From Extreme Points to Object Segmentationcited by 127
  9. 2016Query-Efficient Imitation Learning for End-to-End Autonomous Drivingcited by 83
  10. 2015Learning to Segment Object Candidatescited by 79
  11. 2016RGB-D-based Action Recognition Datasets: A Surveycited by 14
  12. 2016The ACRV Picking Benchmark (APB): A Robotic Shelf Picking Benchmark to Foster Reproducible Researchcited by 11
  13. 2018One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RLcited by 11
  14. 2016Human Action Localization with Sparse Spatial Supervisioncited by 8

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 3; 2022: 2; 2023: 1; 2024: 2; 2025: 1.

10 papers.

  1. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  2. 2021RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learningcited by 31
  3. 2023LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotioncited by 31
  4. 2022Dungeons and Data: A Large-Scale NetHack Datasetcited by 16
  5. 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 14
  6. 2021The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulationcited by 11
  7. 2020DERAIL: Diagnostic Environments for Reward And Imitation Learningcited by 6
  8. 2024Visual IRL for Human-Like Robotic Manipulationcited by 1
  9. 2025TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperationcited by 1
  10. 2022Entropy-driven Unsupervised Keypoint Representation Learning in Videoscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
  2. 2024VITAL: Interactive Few-Shot Imitation Learning via Visual Human-in-the-Loop Corrections
  3. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Play
  4. 2021Imitation Learning: Progress, Taxonomies and Challenges
  5. 2025RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments
  6. 2024Imitation Learning Datasets: A Toolkit For Creating Datasets, Training Agents and Benchmarking
  7. 2025NIL: No-data Imitation Learning by Leveraging Pre-trained Video Diffusion Models
  8. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation