mlsanityData to 7 Oct 2026

Delay-Aware Model-Based Reinforcement Learning for Continuous Control

arXiv 2005.05440 ↗First posted 11 May 2020cs.LG cs.AI stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Action delays degrade the performance of reinforcement learning in many real-world systems. This paper proposes a formal definition of delay-aware Markov Decision Process and proves it can be transformed into standard MDP with augmented states using the Markov reward process. We develop a delay-aware model-based reinforcement learning framework that can incorporate the multi-step delay into the learned system models without learning effort. Experiments with the Gym and MuJoCo platforms show that the proposed delay-aware model-based algorithm is more efficient in training and transferable between systems with various durations of delay compared with off-policy model-free reinforcement learning methods. Codes available at: https://github.com/baimingc/dambrl.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 5; 2024: 8; 2025: 6; 2026: 6.

33 papers.

Sort:

By area:

  1. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  2. 2021Acting in Delayed Environments with Non-Stationary Markov Policiescited by 28
  3. 2022Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Trainingcited by 26
  4. 2021Revisiting State Augmentation methods for Reinforcement Learning with Stochastic Delayscited by 20
  5. 2023Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Explorationcited by 17
  6. 2022Delayed Reinforcement Learning by Imitationcited by 15
  7. 2024Reinforcement Learning from Delayed Observations via World Modelscited by 13
  8. 2025Real-Time Reasoning Agents in Evolving Environmentscited by 13
  9. 2023Neural Laplace Control for Continuous-time Delayed Systemscited by 10
  10. 2024Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delayscited by 10
  11. 2022DACOM: Learning Delay-Aware Communication for Multi-Agent Reinforcement Learningcited by 9
  12. 2024Variational Delayed Policy Optimizationcited by 9
21 more
  1. 2024DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delayscited by 6
  2. 2023Reducing the Cost of Cycle-Time Tuning for Real-World Policy Optimizationcited by 4
  3. 2023DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Controlcited by 4
  4. 2021Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific researchcited by 1
  5. 2022Temporally Layered Architecture for Adaptive, Distributed and Continuous Controlcited by 1
  6. 2023Adaptive PD Control using Deep Reinforcement Learning for Local-Remote Teleoperation with Stochastic Time Delayscited by 1
  7. 2024Control in Stochastic Environment with Delays: A Model-based Reinforcement Learning Approachcited by 1
  8. 2025Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimizationcited by 1
  9. 2025Reinforcement Learning via Conservative Agent for Environments with Random Delayscited by 1
  10. 2026Reinforcement Learning for Control Systems with Time Delays: A Comprehensive Surveycited by 1
  11. 2024Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Controlcited by 0
  12. 2024Patrol Security Game: Defending Against Adversary with Freedom in Attack Timing, Location, and Durationcited by 0
  13. 2024Inverse Delayed Reinforcement Learningcited by 0
  14. 2025Rainbow Delay Compensation: A Multi-Agent Reinforcement Learning Framework for Mitigating Delayed Observationcited by 0
  15. 2025Intelligent Spectrum Management in Satellite Communicationscited by 0
  16. 2025RT-HCP: Dealing with Inference Delays and Sample Efficiency to Learn Directly on Robotic Platformscited by 0
  17. 2026State-Action Inpainting Diffuser for Continuous Control with Delaycited by 0
  18. 2026Delayed homomorphic reinforcement learning for environments with delayed feedbackcited by 0
  19. 2026Structural Equivalence and Learning Dynamics in Delayed MARLcited by 0
  20. 2026Decoupled Delay Compensation: Enhancing Pre-trained MARL Policies via Learned Dynamics Filteringcited by 0
  21. 2026RAPAC-DP: Response-Aligned Pending-Action Compensation for Diffusion Policies under Delayed Executioncited by 0