mlsanityData to 7 Oct 2026

Asynchronous Methods for Deep Reinforcement Learning

arXiv 1602.01783 ↗First posted 4 Feb 2016cs.LGcited by 2,218

Not on the map. Why?

Cited by 83 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a conceptually simple and lightweight framework for deep reinforcement learning that uses asynchronous gradient descent for optimization of deep neural network controllers. We present asynchronous variants of four standard reinforcement learning algorithms and show that parallel actor-learners have a stabilizing effect on training allowing all four methods to successfully train neural network controllers. The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU. Furthermore, we show that asynchronous actor-critic succeeds on a wide variety of continuous motor control problems as well as on a new task of navigating random 3D mazes using a visual input.

Key facts. Cited by 2,218 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Feb 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 2; 2018: 8; 2019: 7; 2020: 8; 2021: 496; 2022: 400; 2023: 387; 2024: 369; 2025: 322; 2026: 219.

2,218 papers.

Sort:

By area:

  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  3. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  4. 2022Red Teaming Language Models with Language Modelscited by 983
  5. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  6. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  7. 2023A Comprehensive Overview of Large Language Modelscited by 639
  8. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  9. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  10. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  11. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  12. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
38 more
  1. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  2. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  3. 2023Augmented Language Models: a Surveycited by 398
  4. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  5. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  6. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  7. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  8. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  9. 2022Transformers are Sample-Efficient World Modelscited by 317
  10. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  11. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  12. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  13. 2022Teaching language models to support answers with verified quotescited by 290
  14. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  15. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  16. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  17. 2025Learning to Reason under Off-Policy Guidancecited by 266
  18. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  19. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  20. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  21. 2021Mastering Atari Games with Limited Datacited by 235
  22. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  23. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  24. 2022Multi-Game Decision Transformerscited by 212
  25. 2022Online Decision Transformercited by 194
  26. 2024Disentangling Length from Quality in Direct Preference Optimizationcited by 194
  27. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  28. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  29. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 186
  30. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 176
  31. 2023Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraintscited by 174
  32. 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Designcited by 163
  33. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  34. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  35. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  36. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  37. 2018Self-Imitation Learningcited by 135
  38. 2021The Benchmark Lotterycited by 133

50 most cited of 2,218 All 2,218 (JSON)