# Control-Optimized Deep Reinforcement Learning for Artificially Intelligent Autonomous Systems

arXiv: https://arxiv.org/abs/2507.00268 · HTML: https://mlsanity.com/paper/2507.00268/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey](https://mlsanity.com/paper/2005.00935/) (2020)
- [Control of a Quadrotor with Reinforcement Learning](https://mlsanity.com/paper/1707.05110/) (2017)
- [PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning](https://mlsanity.com/paper/2001.05012/) (2020)
- [Learning in Restless Multi-Armed Bandits via Adaptive Arm Sequencing Rules](https://mlsanity.com/paper/1906.08120/) (2019)
- [Policy Design for Active Sequential Hypothesis Testing using Deep Learning](https://mlsanity.com/paper/1810.04859/) (2018)
- [Distributed Learning over Markovian Fading Channels for Stable Spectrum Access](https://mlsanity.com/paper/2101.11292/) (2021)
- [Deep Actor-Critic Reinforcement Learning for Anomaly Detection](https://mlsanity.com/paper/1908.10755/) (2019)
- [Learning in Restless Bandits under Exogenous Global Markov Process](https://mlsanity.com/paper/2112.09484/) (2021)
- [Deep Reinforcement Learning for Simultaneous Sensing and Channel Access in Cognitive Networks](https://mlsanity.com/paper/2110.14541/) (2021)
- [Client Selection for Generalization in Accelerated Federated Learning: A Multi-Armed Bandit Approach](https://mlsanity.com/paper/2303.10373/) (2023)
- [Multi-Flow Transmission in Wireless Interference Networks: A Convergent Graph Learning Approach](https://mlsanity.com/paper/2303.15544/) (2023)
- [SINR-Aware Deep Reinforcement Learning for Distributed Dynamic Channel Allocation in Cognitive Interference Networks](https://mlsanity.com/paper/2402.17773/) (2024)
- [Deep Multi-Agent Reinforcement Learning for Decentralized Active Hypothesis Testing](https://mlsanity.com/paper/2309.08477/) (2023)
- [Learning Control for Air Hockey Striking using Deep Reinforcement Learning](https://mlsanity.com/paper/1702.08074/) (2017)
- [Reduced-Dimensional Reinforcement Learning Control using Singular Perturbation Approximations](https://mlsanity.com/paper/2004.14501/) (2020)
- [An Online Learning Approach to Shortest Path and Backpressure Routing in Wireless Networks](https://mlsanity.com/paper/2204.03620/) (2022)
- [Using Deep Reinforcement Learning for mmWave Real-Time Scheduling](https://mlsanity.com/paper/2210.01423/) (2022)

## Abstract (arXiv)

Deep reinforcement learning (DRL) has become a powerful tool for complex decision-making in machine learning and AI. However, traditional methods often assume perfect action execution, overlooking the uncertainties and deviations between an agent's selected actions and the actual system response. In real-world applications, such as robotics, mechatronics, and communication networks, execution mismatches arising from system dynamics, hardware constraints, and latency can significantly degrade performance. This work advances AI by developing a novel control-optimized DRL framework that explicitly models and compensates for action execution mismatches, a challenge largely overlooked in existing methods. Our approach establishes a structured two-stage process: determining the desired action and selecting the appropriate control signal to ensure proper execution. It trains the agent while accounting for action mismatches and controller corrections. By incorporating these factors into the training process, the AI agent optimizes the desired action with respect to both the actual control signal and the intended outcome, explicitly considering execution errors. This approach enhances robustness, ensuring that decision-making remains effective under real-world uncertainties. Our approach offers a substantial advancement for engineering practice by bridging the gap between idealized learning and real-world implementation. It equips intelligent agents operating in engineering environments with the ability to anticipate and adjust for actuation errors and system disturbances during training. We evaluate the framework in five widely used open-source mechanical simulation environments we restructured and developed to reflect real-world operating conditions, showcasing its robustness against uncertainties and offering a highly practical and efficient solution for control-oriented applications.