# Solving Challenging Dexterous Manipulation Tasks With Trajectory Optimisation and Reinforcement Learning

arXiv: https://arxiv.org/abs/2009.05104 · HTML: https://mlsanity.com/paper/2009.05104/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2020.

## Built on by (most cited first, 13 of 13)

- [Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning](https://mlsanity.com/paper/2206.08686/) (2022)
- [Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning](https://mlsanity.com/paper/2111.03062/) (2021)
- [Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation](https://mlsanity.com/paper/2309.00987/) (2023)
- [RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.04150/) (2023)
- [Lessons from Learning to Spin "Pens"](https://mlsanity.com/paper/2407.18902/) (2024)
- [DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands](https://mlsanity.com/paper/2310.08809/) (2023)
- [Learning to Transfer In-Hand Manipulations Using a Greedy Shape Curriculum](https://mlsanity.com/paper/2303.12726/) (2023)
- [D-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous Deformable Manipulation](https://mlsanity.com/paper/2403.12861/) (2024)
- [Geometric Red-Teaming for Robotic Manipulation](https://mlsanity.com/paper/2509.12379/) (2025)
- [Value Summation: A Novel Scoring Function for MPC-based Model-based Reinforcement Learning](https://mlsanity.com/paper/2209.08169/) (2022)
- [Policy Search using Dynamic Mirror Descent MPC for Model Free Off Policy RL](https://mlsanity.com/paper/2110.12239/) (2021)
- [DexHoldem: Playing Texas Hold'em with Dexterous Embodied System](https://mlsanity.com/paper/2605.18727/) (2026)
- [Structured Representation Learning with Locally Linear Embeddings and Adaptive Feature Fusion](https://mlsanity.com/paper/2606.18469/) (2026)

## Abstract (arXiv)

Training agents to autonomously learn how to use anthropomorphic robotic hands has the potential to lead to systems capable of performing a multitude of complex manipulation tasks in unstructured and uncertain environments. In this work, we first introduce a suite of challenging simulated manipulation tasks that current reinforcement learning and trajectory optimisation techniques find difficult. These include environments where two simulated hands have to pass or throw objects between each other, as well as an environment where the agent must learn to spin a long pen between its fingers. We then introduce a simple trajectory optimisation that performs significantly better than existing methods on these environments. Finally, on the challenging PenSpin task we combine sub-optimal demonstrations generated through trajectory optimisation with off-policy reinforcement learning, obtaining performance that far exceeds either of these approaches individually, effectively solving the environment. Videos of all of our results are available at: https://dexterous-manipulation.github.io/