mlsanityData to 14 Oct 2026

Revisiting Implicit Differentiation for Learning Problems in Optimal Control

arXiv 2310.14468 ↗First posted 23 Oct 2023cs.LG cs.RO cs.SYcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper proposes a new method for differentiating through optimal trajectories arising from non-convex, constrained discrete-time optimal control (COC) problems using the implicit function theorem (IFT). Previous works solve a differential Karush-Kuhn-Tucker (KKT) system for the trajectory derivative, and achieve this efficiently by solving an auxiliary Linear Quadratic Regulator (LQR) problem. In contrast, we directly evaluate the matrix equations which arise from applying variable elimination on the Lagrange multiplier terms in the (differential) KKT system. By appropriately accounting for the structure of the terms within the resulting equations, we show that the trajectory derivatives scale linearly with the number of timesteps. Furthermore, our approach allows for easy parallelization, significantly improved scalability with model size, direct computation of vector-Jacobian products and improved numerical stability compared to prior works. As an additional contribution, we unify prior works, addressing claims that computing trajectory derivatives using IFT scales quadratically with the number of timesteps. We evaluate our method on a both synthetic benchmark and four challenging, learning from demonstration benchmarks including a 6-DoF maneuvering quadrotor and 6-DoF rocket powered landing.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 22 of its 49 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

22 of 49 references.

References · 22

  1. 2021Convex optimizationcited by 2,515
  2. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  3. 2019Differentiable Convex Optimization Layerscited by 465
  4. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  5. 2019Differentiation of Blackbox Combinatorial Solverscited by 226
  6. 2020Fast Differentiable Sorting and Rankingcited by 197
  7. 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
  8. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  9. 2019Deep Declarative Networks: A New Hopecited by 78
  10. 2018Differentiable Dynamic Programming for Structured Prediction and Attentioncited by 58
  11. 2019Pontryagin Differentiable Programming: An End-to-End Learning and Control Frameworkcited by 52
  12. 2020Solving the Blind Perspective-n-Point Problem End-To-End With Robust Differentiable Geometric Optimizationcited by 38
10 more
  1. 2023iPlanner: Imperative Path Planningcited by 37
  2. 2019Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Controlcited by 34
  3. 2019Differentiable Gaussian Process Motion Planningcited by 28
  4. 2021Safe Pontryagin Differentiable Programmingcited by 26
  5. 2021Differentiable Factor Graph Optimization for Learning Smootherscited by 17
  6. 2017Generalized Rank Pooling for Activity Recognitioncited by 10
  7. 2021LEO: Learning Energy-based Models in Factor Graph Optimizationcited by 8
  8. 2023Deep Declarative Dynamic Time Warping for End-to-End Learning of Alignment Pathscited by 5
  9. 2018Inverse Optimal Control from Incomplete Trajectory Observationscited by 4
  10. 2019A Differentiable Augmented Lagrangian Method for Bilevel Nonlinear Optimizationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 3.

8 papers.

  1. 2024DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learningcited by 6
  2. 2025End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagationcited by 3
  3. 2024Shadow Program Inversion with Differentiable Planning: A Framework for Unified Robot Program Parameter and Trajectory Optimizationcited by 1
  4. 2024Learning Set Functions with Implicit Differentiationcited by 1
  5. 2025Accelerating Optimization via Differentiable Stopping Timecited by 1
  6. 2026On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonianscited by 1
  7. 2026Neural-Surrogate Co-Design of Aerodynamic Shape and Controlcited by 0
  8. 2026Explaining Process Control Optimisation Recommendations via GradientSHAP and Implicit Differentiationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Gauss-Newton-Induced Structure-Exploiting Algorithm for Differentiable Optimal Control ↗
  2. 2022Differentiable Optimal Control via Differential Dynamic Programming
  3. 2025On the Robustness of Derivative-free Methods for Linear Quadratic Regulator
  4. 2025Bridging Continuous-time LQR and Reinforcement Learning via Gradient Flow of the Bellman Error
  5. 2024DiffuSolve: Diffusion-based Solver for Non-convex Trajectory Optimization
  6. 2026Gradient Dominance in the Linear Quadratic Regulator: A Unified Analysis for Continuous-Time and Discrete-Time Systems
  7. 2025Tailoring Reproducing Kernels for Optimal Control via Policy Iteration
  8. 2026Accelerating trajectory optimization with Sobolev-trained diffusion policies