# Distributed Deep Q-Learning

arXiv: https://arxiv.org/abs/1508.04186 · HTML: https://mlsanity.com/paper/1508.04186/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Aug 2015.

## Built on by (most cited first, 9 of 9)

- [PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference](https://mlsanity.com/paper/2410.21966/) (2024)
- [Survey of Recent Multi-Agent Reinforcement Learning Algorithms Utilizing Centralized Training](https://mlsanity.com/paper/2107.14316/) (2021)
- [Network Support for High-performance Distributed Machine Learning](https://mlsanity.com/paper/2102.03394/) (2021)
- [Efficient Distributed DNNs in the Mobile-edge-cloud Continuum](https://mlsanity.com/paper/2202.11349/) (2022)
- [Scaling up Stochastic Gradient Descent for Non-convex Optimisation](https://mlsanity.com/paper/2210.02882/) (2022)
- [Deep Reinforcement Learning for Decentralized Multi-Robot Control: A DQN Approach to Robustness and Information Integration](https://mlsanity.com/paper/2408.11339/) (2024)
- [Parallelized Reverse Curriculum Generation](https://mlsanity.com/paper/2108.02128/) (2021)
- [Human-Level Control without Server-Grade Hardware](https://mlsanity.com/paper/2111.01264/) (2021)
- [An Efficient Approach for Cooperative Multi-Agent Learning Problems](https://mlsanity.com/paper/2504.04850/) (2025)

## Abstract (arXiv)

We propose a distributed deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. The model is based on the deep Q-network, a convolutional neural network trained with a variant of Q-learning. Its input is raw pixels and its output is a value function estimating future rewards from taking an action given a system state. To distribute the deep Q-network training, we adapt the DistBelief software framework to the context of efficiently training reinforcement learning agents. As a result, the method is completely asynchronous and scales well with the number of machines. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to achieve reasonable success on a simple game with minimal parameter tuning.