# UniBYD: A Unified Framework for Learning Robotic Manipulation Across Embodiments Beyond Imitation of Human Demonstrations arXiv: https://arxiv.org/abs/2512.11609 ยท HTML: https://mlsanity.com/paper/2512.11609/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. - Builds on 37 of its 61 references that are in the map. - First posted on arXiv 12 Dec 2025. ## Placed in - Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/) ## Builds on (most cited first, 30 of 37) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021) - [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017) - [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023) - [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020) - [Latent Action Pretraining from Videos](https://mlsanity.com/paper/2410.11758/) (2024) - [EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video](https://mlsanity.com/paper/2505.11709/) (2025) - [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021) - [Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos](https://mlsanity.com/paper/2507.15597/) (2025) - [Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost](https://mlsanity.com/paper/1810.06045/) (2018) - [Rotating without Seeing: Towards In-hand Dexterity through Touch](https://mlsanity.com/paper/2303.10880/) (2023) - [OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion](https://mlsanity.com/paper/2403.19417/) (2024) - [ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning](https://mlsanity.com/paper/2503.21860/) (2025) - [Learning Generalizable Dexterous Manipulation from Human Grasp Affordance](https://mlsanity.com/paper/2204.02320/) (2022) - [Object-Centric Dexterous Manipulation from Human Motion Data](https://mlsanity.com/paper/2411.04005/) (2024) - [DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation](https://mlsanity.com/paper/2505.24853/) (2025) - [Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models](https://mlsanity.com/paper/2102.06794/) (2021) - [Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration](https://mlsanity.com/paper/2504.12609/) (2025) - [Dex1B: Learning with 1B Demonstrations for Dexterous Manipulation](https://mlsanity.com/paper/2506.17198/) (2025) - [You Only Teach Once: Learn One-Shot Bimanual Robotic Manipulation from Video Demonstrations](https://mlsanity.com/paper/2501.14208/) (2025) - [HERMES: Human-to-Robot Embodied Learning from Multi-Source Motion Data for Mobile Dexterous Manipulation](https://mlsanity.com/paper/2508.20085/) (2025) - [QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer](https://mlsanity.com/paper/2404.07988/) (2024) - [GelSight Svelte: A Human Finger-shaped Single-camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensing](https://mlsanity.com/paper/2309.10885/) (2023) - [Learning to Transfer Human Hand Skills for Robot Manipulations](https://mlsanity.com/paper/2501.04169/) (2025) - [Dexterous Contact-Rich Manipulation via the Contact Trust Region](https://mlsanity.com/paper/2505.02291/) (2025) - [Contact-Implicit Model Predictive Control for Dexterous In-hand Manipulation: A Long-Horizon and Robust Approach](https://mlsanity.com/paper/2402.18897/) (2024) - [Analyzing Key Objectives in Human-to-Robot Retargeting for Dexterous Manipulation](https://mlsanity.com/paper/2506.09384/) (2025) - [Contact-Implicit Trajectory Optimization using Orthogonal Collocation](https://mlsanity.com/paper/1809.06436/) (2018) - [Multi-finger Manipulation via Trajectory Optimization with Differentiable Rolling and Geometric Constraints](https://mlsanity.com/paper/2408.13229/) (2024) - [DexFlow: A Unified Approach for Dexterous Hand Pose Retargeting and Interaction](https://mlsanity.com/paper/2505.01083/) (2025) ## Built on by (most cited first, 1 of 1) - [E-TTS: A New Embodied Test-Time Scaling Framework for Robotic Manipulation](https://mlsanity.com/paper/2606.27268/) (2026) ## Abstract (arXiv) In embodied intelligence, the embodiment gap between robotic and human hands brings significant challenges for learning from human demonstrations. Although some studies have attempted to bridge this gap using reinforcement learning, they remain confined to merely reproducing human manipulation, resulting in limited task performance. Moreover, current methods struggle to support diverse robotic hand configurations. In this paper, we propose UniBYD, a unified framework that uses a dynamic reinforcement learning algorithm to discover manipulation policies aligned with the robot's physical characteristics. To enable consistent modeling across diverse robotic hand morphologies, UniBYD incorporates a unified morphological representation (UMR). Building on UMR, we design a dynamic PPO with an annealed reward schedule, enabling reinforcement learning to transition from offline-informed imitation of human demonstrations to online-adaptive exploration of policies better adapted to diverse robotic morphologies, thereby going beyond mere imitation of human hands. To address the severe state drift caused by the incapacity of early-stage policies, we design a hybrid Markov-based shadow engine that provides fine-grained guidance to anchor the imitation within the expert's manifold. To evaluate UniBYD, we propose UniManip, the first benchmark for cross-embodiment manipulation spanning diverse robotic morphologies. Experiments demonstrate a 44.08% average improvement in success rate over the current state-of-the-art. Our project page is https://zhanheng-creator.github.io/UniBYD.