# DexterityGen: Foundation Controller for Unprecedented Dexterity

arXiv: https://arxiv.org/abs/2502.04307 · HTML: https://mlsanity.com/paper/2502.04307/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2025.

## Built on by (most cited first, 29 of 29)

- [Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids](https://mlsanity.com/paper/2502.20396/) (2025)
- [SPIDER: Scalable Physics-Informed Dexterous Retargeting](https://mlsanity.com/paper/2511.09484/) (2025)
- [Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy](https://mlsanity.com/paper/2504.18829/) (2025)
- [DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation](https://mlsanity.com/paper/2509.04441/) (2025)
- [SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation](https://mlsanity.com/paper/2602.16863/) (2026)
- [Geometric Retargeting: A Principled, Ultrafast Neural Hand Retargeting Algorithm](https://mlsanity.com/paper/2503.07541/) (2025)
- [Object-centric 3D Motion Field for Robot Learning from Human Videos](https://mlsanity.com/paper/2506.04227/) (2025)
- [Dex4D: Task-Agnostic Point Track Policy for Zero-Shot Dexterous Manipulation](https://mlsanity.com/paper/2602.15828/) (2026)
- [DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videos](https://mlsanity.com/paper/2510.08475/) (2025)
- [METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Model](https://mlsanity.com/paper/2511.17366/) (2025)
- [OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfer](https://mlsanity.com/paper/2512.08920/) (2025)
- [Scaffolding Dexterous Manipulation with Vision-Language Models](https://mlsanity.com/paper/2506.19212/) (2025)
- [Learning Dexterous Manipulation Skills from Imperfect Simulations](https://mlsanity.com/paper/2512.02011/) (2025)
- [LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition](https://mlsanity.com/paper/2606.11628/) (2026)
- [LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations](https://mlsanity.com/paper/2508.17547/) (2025)
- [A Minimalist Retargeting-Guided Reinforcement Learning Recipe for Dexterous Manipulation](https://mlsanity.com/paper/2607.11874/) (2026)
- [FlashBack: Consistency Model-Accelerated Shared Autonomy](https://mlsanity.com/paper/2505.16892/) (2025)
- [Grasp2Grasp: Vision-Based Dexterous Grasp Translation via Schr"odinger Bridges](https://mlsanity.com/paper/2506.02489/) (2025)
- [Lightning Grasp: High Performance Procedural Grasp Synthesis with Contact Fields](https://mlsanity.com/paper/2511.07418/) (2025)
- [MILE: A Mechanically Isomorphic Hand Exoskeleton and Visuotactile Robotic Hand for Data Collection in Dexterous Manipulation](https://mlsanity.com/paper/2512.00324/) (2025)
- [UniCross: Unified Cross-Skill Dexterous Manipulation Synthesis](https://mlsanity.com/paper/2607.28198/) (2026)
- [Visuo-Acoustic Hand Pose and Contact Estimation](https://mlsanity.com/paper/2508.00852/) (2025)
- [BORA: Bridging Offline Reinforcement Learning and Online Residual Adaptation for Real-World Dexterous VLA Models](https://mlsanity.com/paper/2605.30226/) (2026)
- [Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?](https://mlsanity.com/paper/2606.26428/) (2026)
- [Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience](https://mlsanity.com/paper/2606.27475/) (2026)
- [Tele-Catch: Adaptive Teleoperation for Dexterous Dynamic 3D Object Catching](https://mlsanity.com/paper/2603.28427/) (2026)
- [Mana: Dexterous Manipulation of Articulated Tools](https://mlsanity.com/paper/2606.13677/) (2026)
- [PatternDex: Learning Interaction Patterns to Guide Reinforcement Learning of Bimanual Dexterous Manipulation of Articulated Objects](https://mlsanity.com/paper/2610.04765/) (2026)
- [DexFLEX: Contact-Aware Foundation Controller for Command-Guided Dexterity](https://mlsanity.com/paper/2610.14372/) (2026)

## Abstract (arXiv)

Teaching robots dexterous manipulation skills, such as tool use, presents a significant challenge. Current approaches can be broadly categorized into two strategies: human teleoperation (for imitation learning) and sim-to-real reinforcement learning. The first approach is difficult as it is hard for humans to produce safe and dexterous motions on a different embodiment without touch feedback. The second RL-based approach struggles with the domain gap and involves highly task-specific reward engineering on complex tasks. Our key insight is that RL is effective at learning low-level motion primitives, while humans excel at providing coarse motion commands for complex, long-horizon tasks. Therefore, the optimal solution might be a combination of both approaches. In this paper, we introduce DexterityGen (DexGen), which uses RL to pretrain large-scale dexterous motion primitives, such as in-hand rotation or translation. We then leverage this learned dataset to train a dexterous foundational controller. In the real world, we use human teleoperation as a prompt to the controller to produce highly dexterous behavior. We evaluate the effectiveness of DexGen in both simulation and real world, demonstrating that it is a general-purpose controller that can realize input dexterous manipulation commands and significantly improves stability by 10-100x measured as duration of holding objects across diverse tasks. Notably, with DexGen we demonstrate unprecedented dexterous skills including diverse object reorientation and dexterous tool use such as pen, syringe, and screwdriver for the first time.