mlsanityData to 14 Oct 2026

HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies

arXiv 2603.12243 ↗First posted 12 Mar 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Mastering dexterous manipulation with multi-fingered hands has been a grand challenge in robotics for decades. Despite its potential, the difficulty of collecting high-quality data remains a primary bottleneck for high-precision tasks. While reinforcement learning and simulation-to-real-world transfer offer a promising alternative, the transferred policies often fail for tasks demanding millimeter-scale precision, such as bimanual piano playing. In this work, we introduce HandelBot, a framework that combines a simulation policy and rapid adaptation through a two-stage pipeline. Starting from a simulation-trained policy, we first apply a structured refinement stage to correct spatial alignments by adjusting lateral finger joints based on physical rollouts. Next, we use residual reinforcement learning to autonomously learn fine-grained corrective actions. Through extensive hardware experiments across five recognized songs, we demonstrate that HandelBot can successfully perform precise bimanual piano playing. Our system outperforms direct simulation deployment by a factor of 1.8x and requires only 30 minutes of physical interaction data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 59 of its 72 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

59 of 72 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Learning Dexterous Manipulation Skills from Imperfect Simulations1 Dec
  2. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning16 Oct
  3. 2025Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids17 Aug
  4. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning29 Jan

Other references · 55

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  5. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  6. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  7. 2019Solving Rubik's Cube with a Robot Handcited by 734
  8. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  9. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  10. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 206
  11. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 189
  12. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
38 more
  1. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
  2. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  3. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 118
  4. 2019DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm Systemcited by 114
  5. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  6. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  7. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 79
  8. 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learningcited by 77
  9. 2024OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulationcited by 71
  10. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 70
  11. 2025DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulationcited by 66
  12. 2023Imitation Bootstrapped Reinforcement Learningcited by 64
  13. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 63
  14. 2025ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrationscited by 60
  15. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 55
  16. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 54
  17. 2024DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabricscited by 42
  18. 2025ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learningcited by 41
  19. 2025DexWild: Dexterous Human Interactions for In-the-Wild Robot Policiescited by 38
  20. 2024Efficient Data Collection for Robotic Manipulation via Compositional Generalizationcited by 37
  21. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstrationcited by 34
  22. 2025PyRoki: A Modular Toolkit for Robot Kinematic Optimizationcited by 30
  23. 2024Lessons from Learning to Spin "Pens"cited by 26
  24. 2024Bimanual Dexterity for Complex Taskscited by 25
  25. 2024AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touchcited by 21
  26. 2025DOGlove: Dexterous Manipulation with a Low-Cost Open-Source Haptic Force Feedback Glovecited by 19
  27. 2025DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulationcited by 19
  28. 2025Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrationscited by 19
  29. 2021Towards Learning to Play Piano with Dexterous Hands and Touchcited by 17
  30. 2023DEFT: Dexterous Fine-Tuning for Real-World Hand Policiescited by 17
  31. 2025A Taxonomy for Evaluating Generalist Robot Manipulation Policiescited by 16
  32. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulationcited by 15
  33. 2026SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulationcited by 15
  34. 2024F"urElise: Capturing and Physically Synthesizing Hand Motions of Piano Performancecited by 10
  35. 2025OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfercited by 10
  36. 2024PianoMime: Learning a Generalist, Dexterous Piano Player from Internet Demonstrationscited by 9
  37. 2025Scaffolding Dexterous Manipulation with Vision-Language Modelscited by 9
  38. 2024RoboCrowd: Scaling Robot Data Collection through Crowdsourcingcited by 5

50 most cited of 55 All 55 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Expressive Robotic Pianist: Mastering Complex Piano Repertoire with Graph-Mimic and Musical Dynamics ↗
  2. 2025PANDORA: Diffusion Policy Learning for Dexterous Robotic Piano Playing
  3. 2026DexDrummer: In-Hand, Contact-Rich, and Long-Horizon Dexterous Robot Drumming ↗
  4. 2026MUSIC: Learning Muscle-Driven Dexterous Hand Control
  5. 2026Enforcing Human-like Kinematics in Dexterous Piano Playing via Adversarial Posture Regularization ↗
  6. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning
  7. 2026Play2Perfect: What Matters in Dexterous Play Pretraining for Precise Assembly?
  8. 2025Dexterous Manipulation through Imitation Learning: A Survey