# Transferring Multiple Policies to Hotstart Reinforcement Learning in an Air Compressor Management Problem

arXiv: https://arxiv.org/abs/2301.12820 · HTML: https://mlsanity.com/paper/2301.12820/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 12 of 12)

- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning](https://mlsanity.com/paper/1511.06342/) (2015)
- [Interactive Learning from Policy-Dependent Human Feedback](https://mlsanity.com/paper/1701.06049/) (2017)
- [Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfer](https://mlsanity.com/paper/1609.07088/) (2016)
- [Learning to Navigate in Cities Without a Map](https://mlsanity.com/paper/1804.00168/) (2018)
- [Fast Policy Learning through Imitation and Reinforcement](https://mlsanity.com/paper/1805.10413/) (2018)
- [Off-Policy Shaping Ensembles in Reinforcement Learning](https://mlsanity.com/paper/1405.5358/) (2014)
- [Effective Warm Start for the Online Actor-Critic Reinforcement Learning based mHealth Intervention](https://mlsanity.com/paper/1704.04866/) (2017)
- [The Actor-Advisor: Policy Gradient With Off-Policy Advice](https://mlsanity.com/paper/1902.02556/) (2019)

## Built on by (most cited first, 1 of 1)

- [Critic as Lyapunov function (CALF): a model-free, stability-ensuring agent](https://mlsanity.com/paper/2409.09869/) (2024)

## Abstract (arXiv)

Many instances of similar or almost-identical industrial machines or tools are often deployed at once, or in quick succession. For instance, a particular model of air compressor may be installed at hundreds of customers. Because these tools perform distinct but highly similar tasks, it is interesting to be able to quickly produce a high-quality controller for machine N+1 given the controllers already produced for machines 1..N. This is even more important when the controllers are learned through Reinforcement Learning, as training takes time, energy and other resources. In this paper, we apply Policy Intersection, a Policy Shaping method, to help a Reinforcement Learning agent learn to solve a new variant of a compressors control problem faster, by transferring knowledge from several previously learned controllers. We show that our approach outperforms loading an old controller, and significantly improves performance in the long run.