# Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark

arXiv: https://arxiv.org/abs/2402.13654 · HTML: https://mlsanity.com/paper/2402.13654/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 14 of 14)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress](https://mlsanity.com/paper/2206.01626/) (2022)
- [Reinforcement Learning for UAV Attitude Control](https://mlsanity.com/paper/1804.04154/) (2018)
- [Combining Deep Reinforcement Learning And Local Control For The Acrobot Swing-up And Balance Task](https://mlsanity.com/paper/2012.11663/) (2020)
- [Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem](https://mlsanity.com/paper/2005.04539/) (2020)
- [Reinforcement Learning for Control of Valves](https://mlsanity.com/paper/2012.14668/) (2020)
- [Aiding reinforcement learning for set point control](https://mlsanity.com/paper/2304.10289/) (2023)

## Built on by (most cited first, 1 of 1)

- [Predictive reinforcement learning based adaptive PID controller](https://mlsanity.com/paper/2506.08509/) (2025)

## Abstract (arXiv)

This paper presents a learning-based control strategy for non-linear throttle valves with an asymmetric hysteresis, leading to a near-optimal controller without requiring any prior knowledge about the environment. We start with a carefully tuned Proportional Integrator (PI) controller and exploit the recent advances in Reinforcement Learning (RL) with Guides to improve the closed-loop behavior by learning from the additional interactions with the valve. We test the proposed control method in various scenarios on three different valves, all highlighting the benefits of combining both PI and RL frameworks to improve control performance in non-linear stochastic systems. In all the experimental test cases, the resulting agent has a better sample efficiency than traditional RL agents and outperforms the PI controller.