mlsanityData to 7 Oct 2026

Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots

arXiv 2402.10329 ↗First posted 15 Feb 2024cs.ROcited by 373

Not on the map. Why?

Cited by 86 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Universal Manipulation Interface (UMI) -- a data collection and policy learning framework that allows direct skill transfer from in-the-wild human demonstrations to deployable robot policies. UMI employs hand-held grippers coupled with careful interface design to enable portable, low-cost, and information-rich data collection for challenging bimanual and dynamic manipulation demonstrations. To facilitate deployable policy learning, UMI incorporates a carefully designed policy interface with inference-time latency matching and a relative-trajectory action representation. The resulting learned policies are hardware-agnostic and deployable across multiple robot platforms. Equipped with these features, UMI framework unlocks new robot manipulation capabilities, allowing zero-shot generalizable dynamic, bimanual, precise, and long-horizon behaviors, by only changing the training data for each task. We demonstrate UMI's versatility and efficacy with comprehensive real-world experiments, where policies learned via UMI zero-shot generalize to novel environments and objects when trained on diverse human demonstrations. UMI's hardware and software system is open-sourced at https://umi-gripper.github.io.

Key facts. Cited by 373 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Feb 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 62; 2025: 152; 2026: 159.

373 papers.

Sort:

By area:

  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  2. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  3. 2026Causal World Modeling for Robot Controlcited by 289
  4. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  5. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  6. 2025Unified Video Action Modelcited by 195
  7. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  8. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  9. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  10. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  11. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 164
  12. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
38 more
  1. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  2. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  3. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  4. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
  5. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  6. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  7. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 112
  8. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
  9. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  10. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 104
  11. 2025Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulationcited by 103
  12. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
  13. 2026ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningcited by 94
  14. 2025Controllable Video Generation: A Surveycited by 93
  15. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 84
  16. 2024Manipulate-Anything: Automating Real-World Robots using Vision-Language Modelscited by 82
  17. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  18. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 75
  19. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  20. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 71
  21. 2025MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulationcited by 69
  22. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 68
  23. 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 67
  24. 2025Humanoid Policy ~ Human Policycited by 65
  25. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 63
  26. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 63
  27. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 61
  28. 2025Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learningcited by 57
  29. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  30. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
  31. 2024Affordance-based Robot Manipulation with Flow Matchingcited by 54
  32. 2025Training Strategies for Efficient Embodied Reasoningcited by 53
  33. 2024Streaming Diffusion Policy: Fast Policy Synthesis with Variable Noise Diffusion Modelscited by 51
  34. 2025From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignmentcited by 51
  35. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 49
  36. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 48
  37. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 47
  38. 2024Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Samplingcited by 46

50 most cited of 373 All 373 (JSON)