# PANDORA: Diffusion Policy Learning for Dexterous Robotic Piano Playing

arXiv: https://arxiv.org/abs/2503.14545 · HTML: https://mlsanity.com/paper/2503.14545/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 36 of its 40 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 30 of 36)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [3D Diffuser Actor: Policy Diffusion with 3D Scene Representations](https://mlsanity.com/paper/2402.10885/) (2024)
- [Diffusion Policy Policy Optimization](https://mlsanity.com/paper/2409.00588/) (2024)
- [Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models](https://mlsanity.com/paper/2310.10639/) (2023)
- [Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning](https://mlsanity.com/paper/2310.12921/) (2023)
- [RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback](https://mlsanity.com/paper/2402.03681/) (2024)
- [Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals](https://mlsanity.com/paper/2407.05996/) (2024)
- [Reinforcement Learning with Perturbed Rewards](https://mlsanity.com/paper/1810.01032/) (2018)
- [The Ingredients for Robotic Diffusion Transformers](https://mlsanity.com/paper/2410.10088/) (2024)
- [Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits](https://mlsanity.com/paper/2411.18704/) (2024)
- [Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards](https://mlsanity.com/paper/1906.05841/) (2019)
- [Diffusion Reward: Learning Rewards via Conditional Video Diffusion](https://mlsanity.com/paper/2312.14134/) (2023)
- [EDMP: Ensemble-of-costs-guided Diffusion for Motion Planning](https://mlsanity.com/paper/2309.11414/) (2023)
- [Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulation](https://mlsanity.com/paper/2410.14868/) (2024)
- [RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.04150/) (2023)
- [DNAct: Diffusion Guided Multi-Task 3D Policy Learning](https://mlsanity.com/paper/2403.04115/) (2024)
- [Adam with model exponential moving average is effective for nonconvex optimization](https://mlsanity.com/paper/2405.18199/) (2024)
- [Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks](https://mlsanity.com/paper/2210.15629/) (2022)
- [LDP: A Local Diffusion Planner for Efficient Robot Navigation and Collision Avoidance](https://mlsanity.com/paper/2407.01950/) (2024)
- [Towards Learning to Play Piano with Dexterous Hands and Touch](https://mlsanity.com/paper/2106.02040/) (2021)
- [Learning Reward Functions for Robotic Manipulation by Observing Humans](https://mlsanity.com/paper/2211.09019/) (2022)
- [Diffusion Trajectory-guided Policy for Long-horizon Robot Manipulation](https://mlsanity.com/paper/2502.10040/) (2025)
- [ViSaRL: Visual Reinforcement Learning Guided by Human Saliency](https://mlsanity.com/paper/2403.10940/) (2024)
- [DARE: Diffusion Policy for Autonomous Robot Exploration](https://mlsanity.com/paper/2410.16687/) (2024)

## Built on by (most cited first, 2 of 2)

- [Learning to Play Piano in the Real World](https://mlsanity.com/paper/2503.15481/) (2025)
- [FORGE-Tree: Diffusion-Forcing Tree Search for Long-Horizon Robot Manipulation](https://mlsanity.com/paper/2510.21744/) (2025)

## Abstract (arXiv)

We present PANDORA, a novel diffusion-based policy learning framework designed specifically for dexterous robotic piano performance. Our approach employs a conditional U-Net architecture enhanced with FiLM-based global conditioning, which iteratively denoises noisy action sequences into smooth, high-dimensional trajectories. To achieve precise key execution coupled with expressive musical performance, we design a composite reward function that integrates task-specific accuracy, audio fidelity, and high-level semantic feedback from a large language model (LLM) oracle. The LLM oracle assesses musical expressiveness and stylistic nuances, enabling dynamic, hand-specific reward adjustments. Further augmented by a residual inverse-kinematics refinement policy, PANDORA achieves state-of-the-art performance in the ROBOPIANIST environment, significantly outperforming baselines in both precision and expressiveness. Ablation studies validate the critical contributions of diffusion-based denoising and LLM-driven semantic feedback in enhancing robotic musicianship. Videos available at: https://taco-group.github.io/PANDORA