# One-shot, Offline and Production-Scalable PID Optimisation with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2210.13906 · HTML: https://mlsanity.com/paper/2210.13906/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 7 of 7)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [RLlib: Abstractions for Distributed Reinforcement Learning](https://mlsanity.com/paper/1712.09381/) (2017)
- [Optimal Control of SOAs with Artificial Intelligence for Sub-Nanosecond Optical Switching](https://mlsanity.com/paper/2006.12165/) (2020)
- [Resource Allocation in Disaggregated Data Centre Systems with Reinforcement Learning](https://mlsanity.com/paper/2106.02412/) (2021)

## Abstract (arXiv)

Proportional-integral-derivative (PID) control underlies more than 97% of automated industrial processes. Controlling these processes effectively with respect to some specified set of performance goals requires finding an optimal set of PID parameters to moderate the PID loop. Tuning these parameters is a long and exhaustive process. A method (patent pending) based on deep reinforcement learning is presented that learns a relationship between generic system properties (e.g. resonance frequency), a multi-objective performance goal and optimal PID parameter values. Performance is demonstrated in the context of a real optical switching product of the foremost manufacturer of such devices globally. Switching is handled by piezoelectric actuators where switching time and optical loss are derived from the speed and stability of actuator-control processes respectively. The method achieves a 5× improvement in the number of actuators that fall within the most challenging target switching speed, ≥ 20% improvement in mean switching speed at the same optical loss and ≥ 75% reduction in performance inconsistency when temperature varies between 5 and 73 degrees celcius. Furthermore, once trained (which takes O(hours)), the model generates actuator-unique PID parameters in a one-shot inference process that takes O(ms) in comparison to up to O(week) required for conventional tuning methods, therefore accomplishing these performance improvements whilst achieving up to a 106× speed-up. After training, the method can be applied entirely offline, incurring effectively zero optimisation-overhead in production.