# Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing

arXiv: https://arxiv.org/abs/2409.16629 · HTML: https://mlsanity.com/paper/2409.16629/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2024.

## Built on by (most cited first, 8 of 8)

- [InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2502.20390/) (2025)
- [SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrations](https://mlsanity.com/paper/2505.02094/) (2025)
- [DyTact: Capturing Dynamic Contacts in Hand-Object Manipulation](https://mlsanity.com/paper/2506.03103/) (2025)
- [HandX: Scaling Bimanual Motion and Interaction Generation](https://mlsanity.com/paper/2603.28766/) (2026)
- [UniCross: Unified Cross-Skill Dexterous Manipulation Synthesis](https://mlsanity.com/paper/2607.28198/) (2026)
- [MUSIC: Learning Muscle-Driven Dexterous Hand Control](https://mlsanity.com/paper/2604.23886/) (2026)
- [MaskAdapt: Learning Flexible Motion Adaptation via Mask-Invariant Prior for Physics-Based Characters](https://mlsanity.com/paper/2603.29272/) (2026)
- [Generalized Audio-Driven Synthesis of Precise Drummer Motion](https://mlsanity.com/paper/2608.19055/) (2026)

## Abstract (arXiv)

We present a novel approach to synthesize dexterous motions for physically simulated hands in tasks that require coordination between the control of two hands with high temporal precision. Instead of directly learning a joint policy to control two hands, our approach performs bimanual control through cooperative learning where each hand is treated as an individual agent. The individual policies for each hand are first trained separately, and then synchronized through latent space manipulation in a centralized environment to serve as a joint policy for two-hand control. By doing so, we avoid directly performing policy learning in the joint state-action space of two hands with higher dimensions, greatly improving the overall training efficiency. We demonstrate the effectiveness of our proposed approach in the challenging guitar-playing task. The virtual guitarist trained by our approach can synthesize motions from unstructured reference data of general guitar-playing practice motions, and accurately play diverse rhythms with complex chord pressing and string picking patterns based on the input guitar tabs that do not exist in the references. Along with this paper, we provide the motion capture data that we collected as the reference for policy training. Code is available at: https://pei-xu.github.io/guitar.