# At Human Speed: Deep Reinforcement Learning with Action Delay

arXiv: https://arxiv.org/abs/1810.07286 · HTML: https://mlsanity.com/paper/1810.07286/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2018.

## Built on by (most cited first, 26 of 26)

- [Real-Time Execution of Action Chunking Flow Policies](https://mlsanity.com/paper/2506.07339/) (2025)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [Acting in Delayed Environments with Non-Stationary Markov Policies](https://mlsanity.com/paper/2101.11992/) (2021)
- [Delayed Reinforcement Learning by Imitation](https://mlsanity.com/paper/2205.05569/) (2022)
- [Benchmarking End-to-End Behavioural Cloning on Video Games](https://mlsanity.com/paper/2004.00981/) (2020)
- [Neural Laplace Control for Continuous-time Delayed Systems](https://mlsanity.com/paper/2302.12604/) (2023)
- [Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delays](https://mlsanity.com/paper/2402.03141/) (2024)
- [Variational Delayed Policy Optimization](https://mlsanity.com/paper/2405.14226/) (2024)
- [DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control](https://mlsanity.com/paper/2306.09010/) (2023)
- [DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delays](https://mlsanity.com/paper/2406.03102/) (2024)
- [Handling Delay in Real-Time Reinforcement Learning](https://mlsanity.com/paper/2503.23478/) (2025)
- [Reducing the Cost of Cycle-Time Tuning for Real-World Policy Optimization](https://mlsanity.com/paper/2305.05760/) (2023)
- [A Neural Active Inference Model of Perceptual-Motor Learning](https://mlsanity.com/paper/2211.10419/) (2022)
- [Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latency](https://mlsanity.com/paper/2608.23831/) (2026)
- [Temporally Layered Architecture for Adaptive, Distributed and Continuous Control](https://mlsanity.com/paper/2301.00723/) (2022)
- [Adaptive PD Control using Deep Reinforcement Learning for Local-Remote Teleoperation with Stochastic Time Delays](https://mlsanity.com/paper/2305.16979/) (2023)
- [Control in Stochastic Environment with Delays: A Model-based Reinforcement Learning Approach](https://mlsanity.com/paper/2402.00313/) (2024)
- [Reinforcement Learning via Conservative Agent for Environments with Random Delays](https://mlsanity.com/paper/2507.18992/) (2025)
- [The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective](https://mlsanity.com/paper/2606.07017/) (2026)
- [Trajectory Planning for Teleoperated Space Manipulators Using Deep Reinforcement Learning](https://mlsanity.com/paper/2408.05460/) (2024)
- [Rainbow Delay Compensation: A Multi-Agent Reinforcement Learning Framework for Mitigating Delayed Observation](https://mlsanity.com/paper/2505.03586/) (2025)
- [Delayed homomorphic reinforcement learning for environments with delayed feedback](https://mlsanity.com/paper/2604.03641/) (2026)
- [Structural Equivalence and Learning Dynamics in Delayed MARL](https://mlsanity.com/paper/2605.04345/) (2026)
- [Residual Reinforcement Learning for Robot Teleoperation under Stochastic Delays](https://mlsanity.com/paper/2605.15480/) (2026)
- [LangLoc: "Tell Me What You See"](https://mlsanity.com/paper/2607.05077/) (2026)
- [Separating Decision Time from Decision Quality in the Real-Time Gap of Distilled Deciders: Evidence from a Game and a Conveyor Simulator](https://mlsanity.com/paper/2610.04810/) (2026)

## Abstract (arXiv)

There has been a recent explosion in the capabilities of game-playing artificial intelligence. Many classes of tasks, from video games to motor control to board games, are now solvable by fairly generic algorithms, based on deep learning and reinforcement learning, that learn to play from experience with minimal prior knowledge. However, these machines often do not win through intelligence alone -- they possess vastly superior speed and precision, allowing them to act in ways a human never could. To level the playing field, we restrict the machine's reaction time to a human level, and find that standard deep reinforcement learning methods quickly drop in performance. We propose a solution to the action delay problem inspired by human perception -- to endow agents with a neural predictive model of the environment which "undoes" the delay inherent in their environment -- and demonstrate its efficacy against professional players in Super Smash Bros. Melee, a popular console fighting game.