# Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems

arXiv: https://arxiv.org/abs/2402.17703 · HTML: https://mlsanity.com/paper/2402.17703/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 8 of 8)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [A Historical Perspective of Adaptive Control and Learning](https://mlsanity.com/paper/2108.11336/) (2021)
- [A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning](https://mlsanity.com/paper/2207.02767/) (2022)
- [Adaptive formation motion planning and control of autonomous underwater vehicles using deep reinforcement learning](https://mlsanity.com/paper/2304.00225/) (2023)

## Built on by (most cited first, 1 of 1)

- [Deep Reinforcement Learning in Applied Control: Challenges, Analysis, and Insights](https://mlsanity.com/paper/2507.08196/) (2025)

## Abstract (arXiv)

Deep Reinforcement Learning (DRL) techniques have received significant attention in control and decision-making algorithms. Most applications involve complex decision-making systems, justified by the algorithms' computational power and cost. While model-based versions are emerging, model-free DRL approaches are intriguing for their independence from models, yet they remain relatively less explored in terms of performance, particularly in applied control. This study conducts a thorough performance analysis comparing the data-driven DRL paradigm with a classical state feedback controller, both designed based on the same cost (reward) function of the linear quadratic regulator (LQR) problem. Twelve additional performance criteria are introduced to assess the controllers' performance, independent of the LQR problem for which they are designed. Two Deep Deterministic Policy Gradient (DDPG)-based controllers are developed, leveraging DDPG's widespread reputation. These controllers are aimed at addressing a challenging setpoint tracking problem in a Non-Minimum Phase (NMP) system. The performance and robustness of the controllers are assessed in the presence of operational challenges, including disturbance, noise, initial conditions, and model uncertainties. The findings suggest that the DDPG controller demonstrates promising behavior under rigorous test conditions. Nevertheless, further improvements are necessary for the DDPG controller to outperform classical methods in all criteria. While DRL algorithms may excel in complex environments owing to the flexibility in the reward function definition, this paper offers practical insights and a comparison framework specifically designed to evaluate these algorithms within the context of control engineering.