mlsanityData to 7 Oct 2026

Accelerated Methods for Deep Reinforcement Learning

arXiv 1803.02811 ↗First posted 7 Mar 2018cs.LG cs.AI cs.DCcited by 40

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (RL) has achieved many recent successes, yet experiment turn-around time remains a key bottleneck in research and in practice. We investigate how to optimize existing deep RL algorithms for modern computers, specifically for a combination of CPUs and GPUs. We confirm that both policy gradient and Q-value learning algorithms can be adapted to learn using many parallel simulator instances. We further find it possible to train using batch sizes considerably larger than are standard, without negatively affecting sample complexity or final performance. We leverage these facts to build a unified framework for parallelization that dramatically hastens experiments in both classes of algorithm. All neural network computations use GPUs, accelerating both data collection and training. Our results include using an entire DGX-1 to learn successful strategies in Atari games in mere minutes, using both synchronous and asynchronous algorithms.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 8; 2023: 9; 2024: 4; 2025: 4; 2026: 1.

40 papers.

Sort:

By area:

  1. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  2. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 139
  3. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  4. 2021Scaling Scaling Laws with Board Gamescited by 62
  5. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 54
  6. 2021Training Larger Networks for Deep Reinforcement Learningcited by 37
  7. 2021Podracer architectures for scalable Reinforcement Learningcited by 37
  8. 2023Small batch deep reinforcement learningcited by 35
  9. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  10. 2022Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Sizecited by 20
  11. 2023CFlowNets: Continuous Control with Generative Flow Networkscited by 19
  12. 2025The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networkscited by 12
28 more
  1. 2025Learning Sim-to-Real Humanoid Locomotion in 15 Minutescited by 11
  2. 2021Fast and Data-Efficient Training of Rainbow: an Experimental Study on Ataricited by 10
  3. 2023SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Corescited by 10
  4. 2023minimax: Efficient Baselines for Autocurricula in JAXcited by 9
  5. 2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
  6. 2023Cleanba: A Reproducible and Efficient Distributed Reinforcement Learning Platformcited by 7
  7. 2021Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Trainingcited by 6
  8. 2021Graph augmented Deep Reinforcement Learning in the GameRLand3D environmentcited by 5
  9. 2022Fast Population-Based Reinforcement Learning on a Single Machinecited by 5
  10. 2021Godot Reinforcement Learning Agentscited by 4
  11. 2022Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolboxcited by 4
  12. 2023Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Secondcited by 4
  13. 2023Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learningcited by 3
  14. 2025Investigating Tax Evasion Emergence Using Dual Large Language Model and Deep Reinforcement Learning Powered Agent-based Simulationcited by 3
  15. 2021Deep Learning for Embodied Vision Navigation: A Surveycited by 2
  16. 2022Fast and Data Efficient Reinforcement Learning from Pixels via Non-Parametric Value Approximationcited by 2
  17. 2022MAN: Multi-Action Networks Learningcited by 2
  18. 2022Actively Learning Costly Reward Functions for Reinforcement Learningcited by 2
  19. 2026Performance Variation in Deep Reinforcement Learningcited by 2
  20. 2022Scaling up Stochastic Gradient Descent for Non-convex Optimisationcited by 1
  21. 2023Spreeze: High-Throughput Parallel Reinforcement Learning Frameworkcited by 1
  22. 2024Mixed-Reality Digital Twins: Leveraging the Physical and Virtual Worlds for Hybrid Sim2Real Transition of Multi-Agent Reinforcement Learning Policiescited by 1
  23. 2024SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systemscited by 1
  24. 2024Speeding up Policy Simulation in Supply Chain RLcited by 1
  25. 2021Progressive extension of reinforcement learning action dimension for asymmetric assembly taskscited by 0
  26. 2021Human-Level Control without Server-Grade Hardwarecited by 0
  27. 2022GPU-Accelerated Policy Optimization via Batch Automatic Differentiation of Gaussian Processes for Real-World Controlcited by 0
  28. 2025Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategiescited by 0