# Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrations

arXiv: https://arxiv.org/abs/2410.02477 · HTML: https://mlsanity.com/paper/2410.02477/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 55 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 30 of 46)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023)
- [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021)
- [OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning](https://mlsanity.com/paper/2406.08858/) (2024)
- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System](https://mlsanity.com/paper/2307.04577/) (2023)
- [DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality](https://mlsanity.com/paper/2210.13702/) (2022)
- [UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy](https://mlsanity.com/paper/2303.00938/) (2023)
- [In-Hand Object Rotation via Rapid Motor Adaptation](https://mlsanity.com/paper/2210.04887/) (2022)
- [DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System](https://mlsanity.com/paper/1910.03135/) (2019)
- [UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning](https://mlsanity.com/paper/2304.00464/) (2023)
- [VideoDex: Learning Dexterity from Internet Videos](https://mlsanity.com/paper/2212.04498/) (2022)
- [AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos](https://mlsanity.com/paper/1912.04443/) (2019)
- [DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video](https://mlsanity.com/paper/2202.00164/) (2022)
- [TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding](https://mlsanity.com/paper/2401.08399/) (2024)
- [Learning Dexterous Grasping with Object-Centric Visual Affordances](https://mlsanity.com/paper/2009.01439/) (2020)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [Rotating without Seeing: Towards In-hand Dexterity through Touch](https://mlsanity.com/paper/2303.10880/) (2023)
- [Reinforcement Learning with Videos: Combining Offline Observations with Interaction](https://mlsanity.com/paper/2011.06507/) (2020)
- [Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube](https://mlsanity.com/paper/2202.10448/) (2022)
- [From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation](https://mlsanity.com/paper/2204.12490/) (2022)
- [OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion](https://mlsanity.com/paper/2403.19417/) (2024)
- [LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning](https://mlsanity.com/paper/2309.06440/) (2023)
- [Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation](https://mlsanity.com/paper/2203.13251/) (2022)
- [ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulation](https://mlsanity.com/paper/2309.03891/) (2023)
- [Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations](https://mlsanity.com/paper/2402.14606/) (2024)

## Built on by (most cited first, 15 of 15)

- [Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos](https://mlsanity.com/paper/2507.15597/) (2025)
- [ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning](https://mlsanity.com/paper/2503.21860/) (2025)
- [Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skills](https://mlsanity.com/paper/2503.12533/) (2025)
- [Dexterous Manipulation through Imitation Learning: A Survey](https://mlsanity.com/paper/2504.03515/) (2025)
- [DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation](https://mlsanity.com/paper/2411.18562/) (2024)
- [SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis](https://mlsanity.com/paper/2412.20104/) (2024)
- [Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration](https://mlsanity.com/paper/2509.09671/) (2025)
- [DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulation](https://mlsanity.com/paper/2510.11258/) (2025)
- [DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videos](https://mlsanity.com/paper/2510.08475/) (2025)
- [MEgoHand: Multimodal Egocentric Hand-Object Interaction Motion Generation](https://mlsanity.com/paper/2505.16602/) (2025)
- [Lemmanaid: Neuro-Symbolic Lemma Conjecturing](https://mlsanity.com/paper/2504.04942/) (2025)
- [D-CODA: Diffusion for Coordinated Dual-Arm Data Augmentation](https://mlsanity.com/paper/2505.04860/) (2025)
- [Universal Dexterous Functional Grasping via Demonstration-Editing Reinforcement Learning](https://mlsanity.com/paper/2512.13380/) (2025)
- [AsymDex: Asymmetry and Relative Coordinates for RL-based Bimanual Dexterity](https://mlsanity.com/paper/2411.13020/) (2024)
- [DexMimic: A Generalist Dexterous Hand-Object Tracking Policy via Value-Guided Reset Allocation](https://mlsanity.com/paper/2610.19089/) (2026)

## Abstract (arXiv)

Bimanual dexterous manipulation is a critical yet underexplored area in robotics. Its high-dimensional action space and inherent task complexity present significant challenges for policy learning, and the limited task diversity in existing benchmarks hinders general-purpose skill development. Existing approaches largely depend on reinforcement learning, often constrained by intricately designed reward functions tailored to a narrow set of tasks. In this work, we present a novel approach for efficiently learning diverse bimanual dexterous skills from abundant human demonstrations. Specifically, we introduce BiDexHD, a framework that unifies task construction from existing bimanual datasets and employs teacher-student policy learning to address all tasks. The teacher learns state-based policies using a general two-stage reward function across tasks with shared behaviors, while the student distills the learned multi-task policies into a vision-based policy. With BiDexHD, scalable learning of numerous bimanual dexterous skills from auto-constructed tasks becomes feasible, offering promising advances toward universal bimanual dexterous manipulation. Our empirical evaluation on the TACO dataset, spanning 141 tasks across six categories, demonstrates a task fulfillment rate of 74.59% on trained tasks and 51.07% on unseen tasks, showcasing the effectiveness and competitive zero-shot generalization capabilities of BiDexHD. For videos and more information, visit our project page https://sites.google.com/view/bidexhd.