# Meta-Reinforcement Learning for Adaptive Control of Second Order Systems

arXiv: https://arxiv.org/abs/2209.09301 · HTML: https://mlsanity.com/paper/2209.09301/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [A Survey of Deep Meta-Learning](https://mlsanity.com/paper/2010.03522/) (2020)
- [Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning](https://mlsanity.com/paper/2111.07171/) (2021)
- [Deep Reinforcement Learning for Process Control: A Primer for Beginners](https://mlsanity.com/paper/2004.05490/) (2020)
- [Meta-Reinforcement Learning for the Tuning of PI Controllers: An Offline Approach](https://mlsanity.com/paper/2203.09661/) (2022)
- [A Meta-Reinforcement Learning Approach to Process Control](https://mlsanity.com/paper/2103.14060/) (2021)

## Built on by (most cited first, 1 of 1)

- [Meta-Learning and Meta-Reinforcement Learning -- Tracing the Path towards DeepMind's Adaptive Agent](https://mlsanity.com/paper/2602.19837/) (2026)

## Abstract (arXiv)

Meta-learning is a branch of machine learning which aims to synthesize data from a distribution of related tasks to efficiently solve new ones. In process control, many systems have similar and well-understood dynamics, which suggests it is feasible to create a generalizable controller through meta-learning. In this work, we formulate a meta reinforcement learning (meta-RL) control strategy that takes advantage of known, offline information for training, such as a model structure. The meta-RL agent is trained over a distribution of model parameters, rather than a single model, enabling the agent to automatically adapt to changes in the process dynamics while maintaining performance. A key design element is the ability to leverage model-based information offline during training, while maintaining a model-free policy structure for interacting with new environments. Our previous work has demonstrated how this approach can be applied to the industrially-relevant problem of tuning proportional-integral controllers to control first order processes. In this work, we briefly reintroduce our methodology and demonstrate how it can be extended to proportional-integral-derivative controllers and second order systems.