mlsanityData to 14 Oct 2026

Efficient Parallel Methods for Deep Reinforcement Learning

arXiv 1705.04862 ↗First posted 13 May 2017cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

We propose a novel framework for efficient parallelization of deep reinforcement learning algorithms, enabling these algorithms to learn from multiple actors on a single machine. The framework is algorithm agnostic and can be applied to on-policy, off-policy, value based and policy gradient based algorithms. Given its inherent parallelism, the framework can be efficiently implemented on a GPU, allowing the usage of powerful models while significantly reducing training time. We demonstrate the effectiveness of our framework by implementing an advantage actor-critic algorithm on a GPU, using on-policy experiences and employing synchronous updates. Our algorithm achieves state-of-the-art performance on the Atari domain after only a few hours of training. Our framework thus opens the door for much faster experimentation on demanding problem domains. Our implementation is open-source and is made public at https://github.com/alfredvc/paac

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 9; 2023: 5; 2024: 5; 2025: 3; 2026: 1.

32 papers.

Sort:

By area:

  1. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 182
  2. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  3. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 81
  4. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 58
  5. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 28
  6. 2021Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximationcited by 23
  7. 2021A Survey on Deep Reinforcement Learning for Audio-Based Applicationscited by 17
  8. 2021Communication Efficient Parallel Reinforcement Learningcited by 10
  9. 2023Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximationcited by 9
  10. 2023Social Contract AI: Aligning AI Assistants with Implicit Group Normscited by 9
  11. 2023A Survey on Vulnerability of Federated Learning: A Learning Algorithm Perspectivecited by 9
  12. 2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
20 more
  1. 2022Autonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Reviewcited by 6
  2. 2022Rate-Constrained Remote Contextual Banditscited by 5
  3. 2022Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolboxcited by 4
  4. 2025Enhancing Diversity in Parallel Agents: A Maximum State Entropy Exploration Storycited by 4
  5. 2022Remote Contextual Banditscited by 3
  6. 2021Parallel Actors and Learners: A Framework for Generating Scalable RL Implementationscited by 2
  7. 2022MAN: Multi-Action Networks Learningcited by 2
  8. 2024SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systemscited by 2
  9. 2022Scaling up Stochastic Gradient Descent for Non-convex Optimisationcited by 1
  10. 2024Speeding up Policy Simulation in Supply Chain RLcited by 1
  11. 2024Design Optimization of Nuclear Fusion Reactor through Deep Reinforcement Learningcited by 1
  12. 2024Toward Debugging Deep Reinforcement Learning Programs with RLExplorercited by 1
  13. 2025Beyond Training-time Poisoning: Component-level and Post-training Backdoors in Deep Reinforcement Learningcited by 1
  14. 2021Human-Level Control without Server-Grade Hardwarecited by 0
  15. 2022Accelerating Deep Reinforcement Learning for Digital Twin Network Optimization with Evolutionary Strategiescited by 0
  16. 2022Long N-step Surrogate Stage Reward to Reduce Variances of Deep Reinforcement Learning in Complex Problemscited by 0
  17. 2023Parallel Q-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulationcited by 0
  18. 2023A Review of Deep Reinforcement Learning in Serverless Computing: Function Scheduling and Resource Auto-Scalingcited by 0
  19. 2025Momentum Boosted Episodic Memory for Improving Learning in Long-Tailed RL Environmentscited by 0
  20. 2026K-Myriad: Jump-starting reinforcement learning with unsupervised parallel agentscited by 0