# Differentiable Optimal Control via Differential Dynamic Programming

arXiv: https://arxiv.org/abs/2209.01117 · HTML: https://mlsanity.com/paper/2209.01117/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 2 Sep 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Built on by (most cited first, 3 of 3)

- [Pontryagin Optimal Control via Neural Networks](https://mlsanity.com/paper/2212.14566/) (2022)
- [DiffCoord: Differentiable Coordination for Distributed Multi-Agent Trajectory Optimization](https://mlsanity.com/paper/2509.01630/) (2025)
- [Merging model-based control with multi-agent reinforcement learning for multi-agent cooperative teaming strategies](https://mlsanity.com/paper/2606.06011/) (2026)

## Abstract (arXiv)

Robot design optimization, imitation learning and system identification share a common problem which requires optimization over robot or task parameters at the same time as optimizing the robot motion. To solve these problems, we can use differentiable optimal control for which the gradients of the robot's motion with respect to the parameters are required. We propose a method to efficiently compute these gradients analytically via the differential dynamic programming (DDP) algorithm using sensitivity analysis (SA). We show that we must include second-order dynamics terms when computing the gradients. However, we do not need to include them when computing the motion. We validate our approach on the pendulum and double pendulum systems. Furthermore, we compare against using the derivatives of the iterative linear quadratic regulator (iLQR), which ignores these second-order terms everywhere, on a co-design task for the Kinova arm, where we optimize the link lengths of the robot for a target reaching task. We show that optimizing using iLQR gradients diverges as ignoring the second-order dynamics affects the computation of the derivatives. Instead, optimizing using DDP gradients converges to the same optimum for a range of initial designs allowing our formulation to scale to complex systems.