# Jacta: A Versatile Planner for Learning Dexterous and Whole-body Manipulation

arXiv: https://arxiv.org/abs/2408.01258 · HTML: https://mlsanity.com/paper/2408.01258/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 43 of its 53 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 30 of 43)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Efficient Online Reinforcement Learning with Offline Data](https://mlsanity.com/paper/2302.02948/) (2023)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards](https://mlsanity.com/paper/1707.08817/) (2017)
- [A micro Lie theory for state estimation in robotics](https://mlsanity.com/paper/1812.01537/) (2018)
- [In-Hand Object Rotation via Rapid Motor Adaptation](https://mlsanity.com/paper/2210.04887/) (2022)
- [Data Quality in Imitation Learning](https://mlsanity.com/paper/2306.02437/) (2023)
- [Imitating Task and Motion Planning with Visuomotor Transformers](https://mlsanity.com/paper/2305.16309/) (2023)
- [Imitation Bootstrapped Reinforcement Learning](https://mlsanity.com/paper/2311.02198/) (2023)
- [MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/2212.05698/) (2022)
- [Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-dynamic Contact Models](https://mlsanity.com/paper/2206.10787/) (2022)
- [Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo](https://mlsanity.com/paper/2212.00541/) (2022)
- [A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation](https://mlsanity.com/paper/2103.00946/) (2021)
- [Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments](https://mlsanity.com/paper/2010.11940/) (2020)
- [A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects](https://mlsanity.com/paper/2011.08177/) (2020)
- [Rethinking Optimization with Differentiable Simulation from a Global Perspective](https://mlsanity.com/paper/2207.00167/) (2022)
- [Harnessing Reinforcement Learning for Neural Motion Planning](https://mlsanity.com/paper/1906.00214/) (2019)
- [Learning a Decentralized Multi-arm Motion Planner](https://mlsanity.com/paper/2011.02608/) (2020)
- [Fast Contact-Implicit Model-Predictive Control](https://mlsanity.com/paper/2107.05616/) (2021)
- [Contact Mode Guided Motion Planning for Quasidynamic Dexterous Manipulation in 3D](https://mlsanity.com/paper/2105.14431/) (2021)

## Built on by (most cited first, 8 of 8)

- [Viser: Imperative, Web-based 3D Visualization in Python](https://mlsanity.com/paper/2507.22885/) (2025)
- [Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation](https://mlsanity.com/paper/2412.02676/) (2024)
- [Bipedalism for Quadrupedal Robots: Versatile Loco-Manipulation through Risk-Adaptive Reinforcement Learning](https://mlsanity.com/paper/2507.20382/) (2025)
- [CoorDex: Coordinating Body and Hand Priors for Continuous Dexterous Humanoid Loco-Manipulation](https://mlsanity.com/paper/2606.23680/) (2026)
- [Reinforcement Learning with Lie Group Orientations for Robotics](https://mlsanity.com/paper/2409.11935/) (2024)
- [MPC-Injection: Biasing Off-Policy Locomotion RL Toward Controller-Induced Behavior Basins](https://mlsanity.com/paper/2606.26392/) (2026)
- [Learning Pivoting Manipulation with Force and Vision Feedback Using Optimization-based Demonstrations](https://mlsanity.com/paper/2508.01082/) (2025)
- [Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL](https://mlsanity.com/paper/2608.12063/) (2026)

## Abstract (arXiv)

Robotic manipulation is challenging due to discontinuous dynamics, as well as high-dimensional state and action spaces. Data-driven approaches that succeed in manipulation tasks require large amounts of data and expert demonstrations, typically from humans. Existing planners are restricted to specific systems and often depend on specialized algorithms for using demonstrations. Therefore, we introduce a flexible motion planner tailored to dexterous and whole-body manipulation tasks. Our planner creates readily usable demonstrations for reinforcement learning algorithms, eliminating the need for additional training pipeline complexities. With this approach, we can efficiently learn policies for complex manipulation tasks, where traditional reinforcement learning alone only makes little progress. Furthermore, we demonstrate that learned policies are transferable to real robotic systems for solving complex dexterous manipulation tasks. Project website: https://jacta-manipulation.github.io/