# Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework

arXiv: https://arxiv.org/abs/1912.12970 · HTML: https://mlsanity.com/paper/1912.12970/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2019.

## Built on by (most cited first, 30 of 52)

- [Implicit energy regularization of neural ordinary-differential-equation control](https://mlsanity.com/paper/2103.06525/) (2021)
- [Safe Pontryagin Differentiable Programming](https://mlsanity.com/paper/2105.14937/) (2021)
- [Training Efficient Controllers via Analytic Policy Gradient](https://mlsanity.com/paper/2209.13052/) (2022)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [Revisiting Implicit Differentiation for Learning Problems in Optimal Control](https://mlsanity.com/paper/2310.14468/) (2023)
- [A Pontryagin Perspective on Reinforcement Learning](https://mlsanity.com/paper/2405.18100/) (2024)
- [Adversarially Robust Learning for Security-Constrained Optimal Power Flow](https://mlsanity.com/paper/2111.06961/) (2021)
- [Control of Dual-Sourcing Inventory Systems using Recurrent Neural Networks](https://mlsanity.com/paper/2201.06126/) (2022)
- [DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learning](https://mlsanity.com/paper/2402.05421/) (2024)
- [Differentiable Nonlinear Model Predictive Control](https://mlsanity.com/paper/2505.01353/) (2025)
- [Myriad: a real-world testbed to bridge trajectory optimization and deep learning](https://mlsanity.com/paper/2202.10600/) (2022)
- [Neural Moving Horizon Estimation for Robust Flight Control](https://mlsanity.com/paper/2206.10397/) (2022)
- [Heuristic-free Optimization of Force-Controlled Robot Search Strategies in Stochastic Environments](https://mlsanity.com/paper/2207.07524/) (2022)
- [Differentiable Bilevel Programming for Stackelberg Congestion Games](https://mlsanity.com/paper/2209.07618/) (2022)
- [Neural ODEs as Feedback Policies for Nonlinear Optimal Control](https://mlsanity.com/paper/2210.11245/) (2022)
- [Online Control-Informed Learning](https://mlsanity.com/paper/2410.03924/) (2024)
- [Optimal Control of Probabilistic Dynamics Models via Mean Hamiltonian Minimization](https://mlsanity.com/paper/2504.02543/) (2025)
- [End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagation](https://mlsanity.com/paper/2510.00359/) (2025)
- [Pontryagin Optimal Control via Neural Networks](https://mlsanity.com/paper/2212.14566/) (2022)
- [Neural Time-Reversed Generalized Riccati Equation](https://mlsanity.com/paper/2312.09310/) (2023)
- [A Differential Dynamic Programming Framework for Inverse Reinforcement Learning](https://mlsanity.com/paper/2407.19902/) (2024)
- [DiffCoord: Differentiable Coordination for Distributed Multi-Agent Trajectory Optimization](https://mlsanity.com/paper/2509.01630/) (2025)
- [Physics-informed neural networks via stochastic Hamiltonian dynamics learning](https://mlsanity.com/paper/2111.08108/) (2021)
- [Optimisation of Structured Neural Controller Based on Continuous-Time Policy Gradient](https://mlsanity.com/paper/2201.06262/) (2022)
- [Distributed Differentiable Dynamic Game for Multi-robot Coordination](https://mlsanity.com/paper/2207.08892/) (2022)
- [Stretched and measured neural predictions of complex network dynamics](https://mlsanity.com/paper/2301.04900/) (2023)
- [A Unified Framework for Neural Computation and Learning Over Time](https://mlsanity.com/paper/2409.12038/) (2024)
- [DiffOP: Reinforcement Learning of Optimization-Based Control Policies via Implicit Policy Gradients](https://mlsanity.com/paper/2411.07484/) (2024)
- [Safe Online Control-Informed Learning](https://mlsanity.com/paper/2512.13868/) (2025)
- [On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonians](https://mlsanity.com/paper/2602.00921/) (2026)

## Abstract (arXiv)

This paper develops a Pontryagin Differentiable Programming (PDP) methodology, which establishes a unified framework to solve a broad class of learning and control tasks. The PDP distinguishes from existing methods by two novel techniques: first, we differentiate through Pontryagin's Maximum Principle, and this allows to obtain the analytical derivative of a trajectory with respect to tunable parameters within an optimal control system, enabling end-to-end learning of dynamics, policies, or/and control objective functions; and second, we propose an auxiliary control system in the backward pass of the PDP framework, and the output of this auxiliary control system is the analytical derivative of the original system's trajectory with respect to the parameters, which can be iteratively solved using standard control tools. We investigate three learning modes of the PDP: inverse reinforcement learning, system identification, and control/planning. We demonstrate the capability of the PDP in each learning mode on different high-dimensional systems, including multi-link robot arm, 6-DoF maneuvering quadrotor, and 6-DoF rocket powered landing.