# DiffLoop: Tuning PID controllers by differentiating through the feedback loop

arXiv: https://arxiv.org/abs/2106.10516 · HTML: https://mlsanity.com/paper/2106.10516/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 19 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [A Compositional Object-Based Approach to Learning Physical Dynamics](https://mlsanity.com/paper/1612.00341/) (2016)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Online Control with Adversarial Disturbances](https://mlsanity.com/paper/1902.08721/) (2019)
- [Improper Learning for Non-Stochastic Control](https://mlsanity.com/paper/2001.09254/) (2020)
- [The Nonstochastic Control Problem](https://mlsanity.com/paper/1911.12178/) (2019)
- [Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergence](https://mlsanity.com/paper/1910.09496/) (2019)
- [Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem](https://mlsanity.com/paper/2005.04539/) (2020)
- [Model-Based Policy Search for Automatic Tuning of Multivariate PID Controllers](https://mlsanity.com/paper/1703.02899/) (2017)

## Built on by (most cited first, 5 of 5)

- [Meta-Reinforcement Learning for the Tuning of PI Controllers: An Offline Approach](https://mlsanity.com/paper/2203.09661/) (2022)
- [PC-Gym: Benchmark Environments For Process Control Problems](https://mlsanity.com/paper/2410.22093/) (2024)
- [Control-Informed Reinforcement Learning for Chemical Processes](https://mlsanity.com/paper/2408.13566/) (2024)
- [Machine learning for industrial sensing and control: A survey and practical perspective](https://mlsanity.com/paper/2401.13836/) (2024)
- [Autotuning Bipedal Locomotion MPC with GRFM-Net for Efficient Sim-to-Real Transfer](https://mlsanity.com/paper/2409.15710/) (2024)

## Abstract (arXiv)

Since most industrial control applications use PID controllers, PID tuning and anti-windup measures are significant problems. This paper investigates tuning the feedback gains of a PID controller via back-calculation and automatic differentiation tools. In particular, we episodically use a cost function to generate gradients and perform gradient descent to improve controller performance. We provide a theoretical framework for analyzing this non-convex optimization and establish a relationship between back-calculation and disturbance feedback policies. We include numerical experiments on linear systems with actuator saturation to show the efficacy of this approach.