mlsanityData to 14 Oct 2026

Imitation Learning with Precisely Labeled Human Demonstrations

arXiv 2504.13803 ↗First posted 18 Apr 2025cs.RO cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Within the imitation learning paradigm, training generalist robots requires large-scale datasets obtainable only through diverse curation. Due to the relative ease to collect, human demonstrations constitute a valuable addition when incorporated appropriately. However, existing methods utilizing human demonstrations face challenges in inferring precise actions, ameliorating embodiment gaps, and fusing with frontier generalist robot training pipelines. In this work, building on prior studies that demonstrate the viability of using hand-held grippers for efficient data collection, we leverage the user's control over the gripper's appearance--specifically by assigning it a unique, easily segmentable color--to enable simple and reliable application of the RANSAC and ICP registration method for precise end-effector pose estimation. We show in simulation that precisely labeled human demonstrations on their own allow policies to reach on average 88.1% of the performance of using robot demonstrations, and boost policy performance when combined with robot demonstrations, despite the inherent embodiment gap.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 26 references that are in the map. First posted on arXiv 18 Apr 2025.

Builds on

23 of 26 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learning13 Jan
  2. 2024SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation1 Nov
  3. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations26 Oct
  4. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Play24 Feb
  5. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training30 Sep
  6. 2020Visual Imitation Made Easy11 Aug

Other references · 17

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  3. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  4. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  5. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  6. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  7. 2022Human-to-Robot Imitation in the Wildcited by 161
  8. 2023Learning to Act without Actionscited by 121
  9. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 119
  10. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 107
  11. 2023On Bringing Robots Homecited by 101
  12. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
5 more
  1. 2025Phantom: Training Robots Without Robots Using Only Human Videoscited by 70
  2. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 52
  3. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 50
  4. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 38
  5. 2024DITTO: Demonstration Imitation by Trajectory Transformationcited by 30

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RwoR: Generating Robot Demonstrations from Human Hand Collection for Policy Learning without Robot
  2. 2024VITAL: Interactive Few-Shot Imitation Learning via Visual Human-in-the-Loop Corrections
  3. 2025Generalist Robot Manipulation beyond Action Labeled Data
  4. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration
  5. 2023On Hand-Held Grippers and the Morphological Gap in Human Manipulation Demonstration
  6. 2022Eliciting Compatible Demonstrations for Multi-Human Imitation Learning
  7. 2026Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models
  8. 2026Human-to-Robot Interaction: Learning from Video Demonstration for Robot Imitation