# Deep Reinforcement Learning in Applied Control: Challenges, Analysis, and Insights

arXiv: https://arxiv.org/abs/2507.08196 · HTML: https://mlsanity.com/paper/2507.08196/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 10 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning](https://mlsanity.com/paper/2111.07171/) (2021)
- [WD3: Taming the Estimation Bias in Deep Reinforcement Learning](https://mlsanity.com/paper/2006.12622/) (2020)
- [Intrinsic Motivation in Dynamical Control Systems](https://mlsanity.com/paper/2301.00005/) (2022)
- [Adaptive formation motion planning and control of autonomous underwater vehicles using deep reinforcement learning](https://mlsanity.com/paper/2304.00225/) (2023)
- [On the Standard Performance Criteria for Applied Control Design: PID, MPC or Machine Learning Controller?](https://mlsanity.com/paper/2503.14379/) (2025)
- [Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems](https://mlsanity.com/paper/2402.17703/) (2024)

## Built on by (most cited first, 2 of 2)

- [Privileged observations enable rapid and reliable policy discovery directly in the physical world](https://mlsanity.com/paper/2512.08463/) (2025)
- [Generalist AI Control: Towards Multi-purpose Adaptive Algorithms](https://mlsanity.com/paper/2607.16313/) (2026)

## Abstract (arXiv)

Over the past decade, remarkable progress has been made in adopting deep neural networks to enhance the performance of conventional reinforcement learning. A notable milestone was the development of Deep Q-Networks (DQN), which achieved human-level performance across a range of Atari games, demonstrating the potential of deep learning to stabilise and scale reinforcement learning. Subsequently, extensions to continuous control algorithms paved the way for a new paradigm in control, one that has attracted broader attention than any classical control approach in recent literature. These developments also demonstrated strong potential for advancing data-driven, model-free algorithms for control and for achieving higher levels of autonomy. However, the application of these methods has remained largely confined to simulated and gaming environments, with ongoing efforts to extend them to real-world applications. Before such deployment can be realised, a solid and quantitative understanding of their performance on applied control problems is necessary. This paper conducts a comparative analysis of these approaches on four diverse benchmark problems with implementation results. This analysis offers a scrutinising and systematic evaluation to shed light on the real-world capabilities and limitations of deep reinforcement learning methods in applied control settings.