# STAMP: Differentiable Task and Motion Planning via Stein Variational Gradient Descent

arXiv: https://arxiv.org/abs/2310.01775 · HTML: https://mlsanity.com/paper/2310.01775/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 27 of its 82 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 27 of 27)

- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planning](https://mlsanity.com/paper/1802.08705/) (2018)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [gradSim: Differentiable simulation for system identification and visuomotor control](https://mlsanity.com/paper/2104.02646/) (2021)
- [The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarks](https://mlsanity.com/paper/1401.3839/) (2014)
- [Scalable Differentiable Physics for Learning and Control](https://mlsanity.com/paper/2007.02168/) (2020)
- [Imitating Task and Motion Planning with Visuomotor Transformers](https://mlsanity.com/paper/2305.16309/) (2023)
- [DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting](https://mlsanity.com/paper/2105.12244/) (2021)
- [Online Replanning in Belief Space for Partially Observable Task and Motion Problems](https://mlsanity.com/paper/1911.04577/) (2019)
- [An End-to-End Differentiable Framework for Contact-Aware Robot Design](https://mlsanity.com/paper/2107.07501/) (2021)
- [Message Passing Stein Variational Gradient Descent](https://mlsanity.com/paper/1711.04425/) (2017)
- [FFRob: Leveraging Symbolic Planning for Efficient Task and Motion Planning](https://mlsanity.com/paper/1608.01335/) (2016)
- [Stein Variational Model Predictive Control](https://mlsanity.com/paper/2011.07641/) (2020)
- [Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning](https://mlsanity.com/paper/2110.00792/) (2021)
- [Probabilistic Inference of Simulation Parameters via Parallel Differentiable Simulation](https://mlsanity.com/paper/2109.08815/) (2021)
- [Stein Variational Inference for Discrete Distributions](https://mlsanity.com/paper/2003.00605/) (2020)
- [Extended Tree Search for Robot Task and Motion Planning](https://mlsanity.com/paper/2103.05456/) (2021)
- [Anytime Integrated Task and Motion Policies for Stochastic Environments](https://mlsanity.com/paper/1904.13006/) (2019)
- [Sampling-Based Methods for Factored Task and Motion Planning](https://mlsanity.com/paper/1801.00680/) (2018)
- [Describing Physics For Physical Reasoning: Force-based Sequential Manipulation Planning](https://mlsanity.com/paper/2002.12780/) (2020)
- [Differentiable Physics Simulations with Contacts: Do They Have Correct Gradients w.r.t. Position, Velocity and Control?](https://mlsanity.com/paper/2207.05060/) (2022)
- [Dynamic Movement Primitives: Volumetric Obstacle Avoidance Using Dynamic Potential Functions](https://mlsanity.com/paper/2007.00518/) (2020)
- [Probabilistic Framework for Constrained Manipulations and Task and Motion Planning under Uncertainty](https://mlsanity.com/paper/2003.04259/) (2020)
- [Differentiable Task Assignment and Motion Planning](https://mlsanity.com/paper/2304.09734/) (2023)

## Built on by (most cited first, 4 of 4)

- [Stein Variational Evolution Strategies](https://mlsanity.com/paper/2410.10390/) (2024)
- [Shadow Program Inversion with Differentiable Planning: A Framework for Unified Robot Program Parameter and Trajectory Optimization](https://mlsanity.com/paper/2409.08678/) (2024)
- [What Do You Need for Compositional Generalization in Diffusion Planning?](https://mlsanity.com/paper/2505.18083/) (2025)
- [Constrained Stein Variational Gradient Descent for Robot Perception, Planning, and Identification](https://mlsanity.com/paper/2506.00589/) (2025)

## Abstract (arXiv)

Planning for sequential robotics tasks often requires integrated symbolic and geometric reasoning. TAMP algorithms typically solve these problems by performing a tree search over high-level task sequences while checking for kinematic and dynamic feasibility. This can be inefficient because, typically, candidate task plans resulting from the tree search ignore geometric information. This often leads to motion planning failures that require expensive backtracking steps to find alternative task plans. We propose a novel approach to TAMP called Stein Task and Motion Planning (STAMP) that relaxes the hybrid optimization problem into a continuous domain. This allows us to leverage gradients from differentiable physics simulation to fully optimize discrete and continuous plan parameters for TAMP. In particular, we solve the optimization problem using a gradient-based variational inference algorithm called Stein Variational Gradient Descent. This allows us to find a distribution of solutions within a single optimization run. Furthermore, we use an off-the-shelf differentiable physics simulator that is parallelized on the GPU to run parallelized inference over diverse plan parameters. We demonstrate our method on a variety of problems and show that it can find multiple diverse plans in a single optimization run while also being significantly faster than existing approaches.