mlsanityData to 14 Oct 2026

Reinforcement Learning with Random Delays

arXiv 2010.02966 ↗First posted 6 Oct 2020cs.LGcited by 59

Not on the map. Why?

Cited by 28 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Action and observation delays commonly occur in many Reinforcement Learning applications, such as remote control scenarios. We study the anatomy of randomly delayed environments, and show that partially resampling trajectory fragments in hindsight allows for off-policy multi-step value estimation. We apply this principle to derive Delay-Correcting Actor-Critic (DCAC), an algorithm based on Soft Actor-Critic with significantly better performance in environments with delays. This is shown theoretically and also demonstrated practically on a delay-augmented version of the MuJoCo continuous control benchmark.

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 2; 2023: 12; 2024: 16; 2025: 9; 2026: 17.

59 papers.

Sort:

By area:

  1. 2024Dense Reward for Free in Reinforcement Learning from Human Feedbackcited by 78
  2. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 38
  3. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  4. 2021Revisiting State Augmentation methods for Reinforcement Learning with Stochastic Delayscited by 21
  5. 2021Reinforcement Learning for Feedback-Enabled Cyber Resiliencecited by 19
  6. 2022Delayed Reinforcement Learning by Imitationcited by 16
  7. 2024Reinforcement Learning from Delayed Observations via World Modelscited by 13
  8. 2023Neural Laplace Control for Continuous-time Delayed Systemscited by 10
  9. 2023Decision Transformer under Random Frame Droppingcited by 10
  10. 2024Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delayscited by 10
  11. 2024Variational Delayed Policy Optimizationcited by 9
  12. 2023DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Controlcited by 7
38 more
  1. 2024Stochastic Approximation with Delayed Updates: Finite-Time Rates under Markovian Samplingcited by 7
  2. 2024DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delayscited by 7
  3. 2024Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inferencecited by 7
  4. 2025Handling Delay in Real-Time Reinforcement Learningcited by 7
  5. 2024Deployable Reinforcement Learning with Variable Control Ratecited by 5
  6. 2023Posterior Sampling with Delayed Feedback for Reinforcement Learning with Linear Function Approximationcited by 3
  7. 2024Reinforcement Learning with Elastic Time Stepscited by 3
  8. 2025Model-Based Reinforcement Learning under Random Observation Delayscited by 3
  9. 2026A Reduction from Delayed to Immediate Feedback for Online Convex Optimization with Improved Guaranteescited by 3
  10. 2023Facilitating Sim-to-real by Intrinsic Stochasticity of Real-Time Simulation in Reinforcement Learning for Robot Manipulationcited by 2
  11. 2023Reinforcement Learning with Delayed, Composite, and Partially Anonymous Rewardcited by 2
  12. 2023Multi-Timescale Control and Communications with Deep Reinforcement Learning -- Part I: Communication-Aware Vehicle Controlcited by 2
  13. 2023Episodic Return Decomposition by Difference of Implicitly Assigned Sub-Trajectory Rewardcited by 2
  14. 2024Tree Search-Based Policy Optimization under Stochastic Execution Delaycited by 2
  15. 2024MOSEAC: Streamlined Variable Time Step Reinforcement Learningcited by 2
  16. 2026Finding the Time to Think: Learning Planning Budgets in Real-Time RLcited by 2
  17. 2026Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference Latencycited by 2
  18. 2022Off-Beat Multi-Agent Reinforcement Learningcited by 1
  19. 2023Adaptive PD Control using Deep Reinforcement Learning for Local-Remote Teleoperation with Stochastic Time Delayscited by 1
  20. 2025Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimizationcited by 1
  21. 2025Reinforcement Learning via Conservative Agent for Environments with Random Delayscited by 1
  22. 2025RT-HCP: Dealing with Inference Delays and Sample Efficiency to Learn Directly on Robotic Platformscited by 1
  23. 2025Reinforcement Learning with Action-Triggered Observationscited by 1
  24. 2026Reinforcement Learning for Control Systems with Time Delays: A Comprehensive Surveycited by 1
  25. 2026CALF: Communication-Aware Learning Framework for Distributed Reinforcement Learningcited by 1
  26. 2026Capacity-Constrained Online Convex Optimization with Delayed Feedbackcited by 1
  27. 2026World Models Under Asynchronous Sensor Observationscited by 1
  28. 2023Actor-Director-Critic: A Novel Deep Reinforcement Learning Frameworkcited by 0
  29. 2023Robotic Packaging Optimization with Reinforcement Learningcited by 0
  30. 2023Simple Noisy Environment Augmentation for Reinforcement Learningcited by 0
  31. 2024Delay-Aware Reinforcement Learning for Highway On-Ramp Merging under Stochastic Communication Latencycited by 0
  32. 2024Novel Actor-Critic Algorithm for Robust Decision Making of CAV under Delays and Loss of V2X Datacited by 0
  33. 2024Variable Time Step Reinforcement Learning for Robotic Applicationscited by 0
  34. 2024Trajectory Planning for Teleoperated Space Manipulators Using Deep Reinforcement Learningcited by 0
  35. 2024Application of Soft Actor-Critic Algorithms in Optimizing Wastewater Treatment with Time Delays Integrationcited by 0
  36. 2025Rainbow Delay Compensation: A Multi-Agent Reinforcement Learning Framework for Mitigating Delayed Observationcited by 0
  37. 2025A Survey of Freshness-Aware Wireless Networking with Reinforcement Learningcited by 0
  38. 2026State-Action Inpainting Diffuser for Continuous Control with Delaycited by 0

50 most cited of 59 All 59 (JSON)