Data to 7 Oct 2026

Massively Parallel Methods for Deep Reinforcement Learning

arXiv 1507.04296 ↗First posted 15 Jul 2015cs.LG cs.AI cs.DCcited by 80

Not on the map. Why?

Abstract (arXiv)

We present the first massively distributed architecture for deep reinforcement learning. This architecture uses four main components: parallel actors that generate new behaviour; parallel learners that are trained from stored experience; a distributed neural network to represent the value function or behaviour policy; and a distributed store of experience. We used our architecture to implement the Deep Q-Network algorithm (DQN). Our distributed algorithm was applied to 49 games from Atari 2600 games from the Arcade Learning Environment, using identical hyperparameters. Our performance surpassed non-distributed DQN in 41 of the 49 games and also reduced the wall-time required to achieve these results by an order of magnitude on most games.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jul 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 19; 2023: 13; 2024: 8; 2025: 11; 2026: 4.

80 papers.

Sort:

By area:

2022HyperTree Proof Search for Neural Theorem Provingcited by 192
2021The Benchmark Lotterycited by 133
2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
2022Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delayscited by 61
2021Podracer architectures for scalable Reinforcement Learningcited by 37
2022Chain of Thought Imitation with Procedure Cloningcited by 30
2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 27
2021Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximationcited by 23
2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23

38 more

2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 19
2022Parametrically Retargetable Decision-Makers Tend To Seek Powercited by 19
2024SAPG: Split and Aggregate Policy Gradientscited by 17
2022Uncovering Instabilities in Variational-Quantum Deep Q-Networkscited by 14
2024Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulationcited by 14
2022Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computerscited by 13
2025Trust Region Masking for Long-Horizon LLM Reinforcement Learningcited by 13
2021Communication Efficient Parallel Reinforcement Learningcited by 10
2021Fast and Data-Efficient Training of Rainbow: an Experimental Study on Ataricited by 10
2022Social Network Structure Shapes Innovation: Experience-sharing in RL with SAPIENScited by 10
2023SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Corescited by 10
2021Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learningcited by 9
2021Weakly-Supervised Domain Adaptation of Deep Regression Trackers via Reinforced Knowledge Distillationcited by 9
2023Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximationcited by 9
2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
2025Provably Robust Federated Reinforcement Learningcited by 8
2021A Cram'er Distance perspective on Quantile Regression based Distributional Reinforcement Learningcited by 7
2021Faster Deep Reinforcement Learning with Slower Online Networkcited by 7
2021Decision Making in Monopoly using a Hybrid Deep Reinforcement Learning Approachcited by 6
2023FedHQL: Federated Heterogeneous Q-Learningcited by 6
2021RL-Scope: Cross-Stack Profiling for Deep Reinforcement Learning Workloadscited by 5
2022Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolboxcited by 4
2023Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Secondcited by 4
2024A survey on secure decentralized optimization and learningcited by 4
2023Client Selection for Federated Policy Optimization with Environment Heterogeneitycited by 3
2023Design Principles for Model Generalization and Scalable AI Integration in Radio Access Networkscited by 3
2023VDFD: Multi-Agent Value Decomposition Framework with Disentangled World Modelcited by 3
2023High Throughput Training of Deep Surrogates from Large Ensemble Runscited by 3
2024QEDCartographer: Automating Formal Verification Using Reward-Free Reinforcement Learningcited by 3
2021Coding for Distributed Multi-Agent Reinforcement Learningcited by 2
2021Parallel Actors and Learners: A Framework for Generating Scalable RL Implementationscited by 2
2022One Policy is Enough: Parallel Exploration with a Single Policy is Near-Optimal for Reward-Free Reinforcement Learningcited by 2
2022MAN: Multi-Action Networks Learningcited by 2
2022Actively Learning Costly Reward Functions for Reinforcement Learningcited by 2
2023Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challengescited by 2
2024MAIDCRL: Semi-centralized Multi-Agent Influence Dense-CNN Reinforcement Learningcited by 2
2025Delphos: A reinforcement learning framework for assisting discrete choice model specificationcited by 2
2025Generalization in Reinforcement Learning for Radio Access Networkscited by 2

50 most cited of 80 All 80 (JSON)