# Shadow Program Inversion with Differentiable Planning: A Framework for Unified Robot Program Parameter and Trajectory Optimization

arXiv: https://arxiv.org/abs/2409.08678 · HTML: https://mlsanity.com/paper/2409.08678/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 57 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Scalable Differentiable Physics for Learning and Control](https://mlsanity.com/paper/2007.02168/) (2020)
- [Continuous-Time Gaussian Process Motion Planning via Probabilistic Inference](https://mlsanity.com/paper/1707.07383/) (2017)
- [The Elements of Differentiable Programming](https://mlsanity.com/paper/2403.14606/) (2024)
- [Differentiable Gaussian Process Motion Planning](https://mlsanity.com/paper/1907.09591/) (2019)
- [Safe Pontryagin Differentiable Programming](https://mlsanity.com/paper/2105.14937/) (2021)
- [Encoding Physical Constraints in Differentiable Newton-Euler Algorithm](https://mlsanity.com/paper/2001.08861/) (2020)
- [Practice Makes Perfect: Planning to Learn Skill Parameter Policies](https://mlsanity.com/paper/2402.15025/) (2024)
- [Multimodal Trajectory Optimization for Motion Planning](https://mlsanity.com/paper/2003.07054/) (2020)
- [Skill-based Multi-objective Reinforcement Learning of Industrial Robot Tasks with Planning and Knowledge Integration](https://mlsanity.com/paper/2203.10033/) (2022)
- [Revisiting Implicit Differentiation for Learning Problems in Optimal Control](https://mlsanity.com/paper/2310.14468/) (2023)
- [CALIPSO: A Differentiable Solver for Trajectory Optimization with Conic and Complementarity Constraints](https://mlsanity.com/paper/2205.09255/) (2022)
- [Robot Program Parameter Inference via Differentiable Shadow Program Inversion](https://mlsanity.com/paper/2103.14452/) (2021)
- [Trajectory Optimization with Optimization-Based Dynamics](https://mlsanity.com/paper/2109.04928/) (2021)
- [Metaheuristics "In the Large"](https://mlsanity.com/paper/2011.09821/) (2020)
- [Heuristic-free Optimization of Force-Controlled Robot Search Strategies in Stochastic Environments](https://mlsanity.com/paper/2207.07524/) (2022)
- [Differentiable Task Assignment and Motion Planning](https://mlsanity.com/paper/2304.09734/) (2023)
- [STAMP: Differentiable Task and Motion Planning via Stein Variational Gradient Descent](https://mlsanity.com/paper/2310.01775/) (2023)
- [Local Policy Optimization for Trajectory-Centric Reinforcement Learning](https://mlsanity.com/paper/2001.08092/) (2020)
- [A Unifying Variational Framework for Gaussian Process Motion Planning](https://mlsanity.com/paper/2309.00854/) (2023)
- [MuTT: A Multimodal Trajectory Transformer for Robot Skills](https://mlsanity.com/paper/2407.15660/) (2024)
- [Meta-Learning Parameterized First-Order Optimizers using Differentiable Convex Optimization](https://mlsanity.com/paper/2303.16952/) (2023)

## Built on by (most cited first, 1 of 1)

- [AI-based Framework for Robust Model-Based Connector Mating in Robotic Wire Harness Installation](https://mlsanity.com/paper/2503.09409/) (2025)

## Abstract (arXiv)

This paper presents SPI-DP, a novel first-order optimizer capable of optimizing robot programs with respect to both high-level task objectives and motion-level constraints. To that end, we introduce DGPMP2-ND, a differentiable collision-free motion planner for serial N-DoF kinematics, and integrate it into an iterative, gradient-based optimization approach for generic, parameterized robot program representations. SPI-DP allows first-order optimization of planned trajectories and program parameters with respect to objectives such as cycle time or smoothness subject to e.g. collision constraints, while enabling humans to understand, modify or even certify the optimized programs. We provide a comprehensive evaluation on two practical household and industrial applications.