# DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation

arXiv: https://arxiv.org/abs/2505.21864 · HTML: https://mlsanity.com/paper/2505.21864/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2025.

## Built on by (most cited first, 30 of 66)

- [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications](https://mlsanity.com/paper/2510.07077/) (2025)
- [Being-H0.7: A Latent World-Action Model from Egocentric Videos](https://mlsanity.com/paper/2605.00078/) (2026)
- [Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization](https://mlsanity.com/paper/2601.12993/) (2026)
- [World Action Models: The Next Frontier in Embodied AI](https://mlsanity.com/paper/2605.12090/) (2026)
- [H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulation](https://mlsanity.com/paper/2507.23523/) (2025)
- [Emergence of Human to Robot Transfer in Vision-Language-Action Models](https://mlsanity.com/paper/2512.22414/) (2025)
- [SPIDER: Scalable Physics-Informed Dexterous Retargeting](https://mlsanity.com/paper/2511.09484/) (2025)
- [CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videos](https://mlsanity.com/paper/2601.04061/) (2026)
- [RDT2: Exploring the Scaling Limit of UMI Data Towards Zero-Shot Cross-Embodiment Generalization](https://mlsanity.com/paper/2602.03310/) (2026)
- [Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories](https://mlsanity.com/paper/2607.15330/) (2026)
- [In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Data](https://mlsanity.com/paper/2511.15704/) (2025)
- [A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulation](https://mlsanity.com/paper/2602.01067/) (2026)
- [Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack](https://mlsanity.com/paper/2606.14409/) (2026)
- [DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation](https://mlsanity.com/paper/2509.04441/) (2025)
- [ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrations](https://mlsanity.com/paper/2510.01607/) (2025)
- [OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction](https://mlsanity.com/paper/2512.16842/) (2025)
- [SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation](https://mlsanity.com/paper/2602.16863/) (2026)
- [Humanoid Manipulation Interface: Humanoid Whole-Body Manipulation from Robot-Free Demonstrations](https://mlsanity.com/paper/2602.06643/) (2026)
- [UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policies](https://mlsanity.com/paper/2510.02614/) (2025)
- [OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfer](https://mlsanity.com/paper/2512.08920/) (2025)
- [Learning Dexterous Manipulation Skills from Imperfect Simulations](https://mlsanity.com/paper/2512.02011/) (2025)
- [Dexora: Open-source VLA for High-DoF Bimanual Dexterity](https://mlsanity.com/paper/2605.18722/) (2026)
- [Cross-Hand Latent Representation for Vision-Language-Action Models](https://mlsanity.com/paper/2603.10158/) (2026)
- [Data Pyramid for Embodied Manipulation: A Survey](https://mlsanity.com/paper/2607.24744/) (2026)
- [FACT: Failure-Aware Causal Training for World-Action Models](https://mlsanity.com/paper/2608.10232/) (2026)
- [AoE: Always-on Egocentric Human Video Collection for Embodied AI](https://mlsanity.com/paper/2602.23893/) (2026)
- [A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies](https://mlsanity.com/paper/2604.13645/) (2026)
- [HARP-VLA: Human-Robot Aligned Representation Learning for Vision-Language-Action Model](https://mlsanity.com/paper/2605.31234/) (2026)
- [Human Universal Grasping](https://mlsanity.com/paper/2606.17054/) (2026)
- [Cloak: Zero-Shot Cross-Embodiment Manipulation by Masking the End-Effector from the VLA](https://mlsanity.com/paper/2606.22836/) (2026)

## Abstract (arXiv)

We present DexUMI - a data collection and policy learning framework that uses the human hand as the natural interface to transfer dexterous manipulation skills to various robot hands. DexUMI includes hardware and software adaptations to minimize the embodiment gap between the human hand and various robot hands. The hardware adaptation bridges the kinematics gap using a wearable hand exoskeleton. It allows direct haptic feedback in manipulation data collection and adapts human motion to feasible robot hand motion. The software adaptation bridges the visual gap by replacing the human hand in video data with high-fidelity robot hand inpainting. We demonstrate DexUMI's capabilities through comprehensive real-world experiments on two different dexterous robot hand hardware platforms, achieving an average task success rate of 86%.