# Predictive reinforcement learning based adaptive PID controller

arXiv: https://arxiv.org/abs/2506.08509 · HTML: https://mlsanity.com/paper/2506.08509/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning](https://mlsanity.com/paper/2111.07171/) (2021)
- [Event-Triggered Safe Bayesian Optimization on Quadcopters](https://mlsanity.com/paper/2312.08058/) (2023)
- [System Identification and Control Using Lyapunov-Based Deep Neural Networks without Persistent Excitation: A Concurrent Learning Approach](https://mlsanity.com/paper/2505.10678/) (2025)
- [Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark](https://mlsanity.com/paper/2402.13654/) (2024)
- [Controlling dynamics of stochastic systems with deep reinforcement learning](https://mlsanity.com/paper/2502.18111/) (2025)

## Abstract (arXiv)

Purpose: This study aims to address the challenges of controlling unstable and nonlinear systems by proposing an adaptive PID controller based on predictive reinforcement learning (PRL-PID), where the PRL-PID combines the advantages of both data-driven and model-driven approaches. Design/methodology/approach: A predictive reinforcement learning framework is introduced, incorporating action smooth strategy to suppress overshoot and oscillations, and a hierarchical reward function to support training. Findings: Experimental results show that the PRL-PID controller achieves superior stability and tracking accuracy in nonlinear, unstable, and strongly coupled systems, consistently outperforming existing RL-tuned PID methods while maintaining excellent robustness and adaptability across diverse operating conditions. Originality/Value: By adopting predictive learning, the proposed PRL-PID integrates system model priors into data-driven control, enhancing both the control framework's training efficiency and the controller's stability. As a result, PRL-PID provides a balanced blend of model-based and data-driven approaches, delivering robust, high-performance control.