# Implementing the Deep Q-Network

arXiv: https://arxiv.org/abs/1711.07478 · HTML: https://mlsanity.com/paper/1711.07478/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2017.

## Built on by (most cited first, 10 of 10)

- [A review on data-driven constitutive laws for solids](https://mlsanity.com/paper/2405.03658/) (2024)
- [Polycraft World AI Lab (PAL): An Extensible Platform for Evaluating Artificial Intelligence Agents](https://mlsanity.com/paper/2301.11891/) (2023)
- [Multi-Agent Reinforcement Learning for Dynamic Pricing in Supply Chains: Benchmarking Strategic Agent Behaviours under Realistically Simulated Market Conditions](https://mlsanity.com/paper/2507.02698/) (2025)
- [Human-Level Control without Server-Grade Hardware](https://mlsanity.com/paper/2111.01264/) (2021)
- [Empowering NLG: Offline Reinforcement Learning for Informal Summarization in Online Domains](https://mlsanity.com/paper/2306.17174/) (2023)
- [Comparing Multiclass Classification Algorithms for Financial Distress Prediction](https://mlsanity.com/paper/2307.03908/) (2023)
- [Trip Planning for Autonomous Vehicles with Wireless Data Transfer Needs Using Reinforcement Learning](https://mlsanity.com/paper/2309.12534/) (2023)
- [Optimizing Conversational Product Recommendation via Reinforcement Learning](https://mlsanity.com/paper/2507.01060/) (2025)
- [Reinforcement Learning Based Traffic Signal Design to Minimize Queue Lengths](https://mlsanity.com/paper/2509.21745/) (2025)
- [SIGMA: Symmetry-aware, Intelligent, Geometric, Multi-faced Adaptive Traffic Control for Reliable Intelligent Transportation Systems](https://mlsanity.com/paper/2608.18263/) (2026)

## Abstract (arXiv)

The Deep Q-Network proposed by Mnih et al. [2015] has become a benchmark and building point for much deep reinforcement learning research. However, replicating results for complex systems is often challenging since original scientific publications are not always able to describe in detail every important parameter setting and software engineering solution. In this paper, we present results from our work reproducing the results of the DQN paper. We highlight key areas in the implementation that were not covered in great detail in the original paper to make it easier for researchers to replicate these results, including termination conditions and gradient descent algorithms. Finally, we discuss methods for improving the computational performance and provide our own implementation that is designed to work with a range of domains, and not just the original Arcade Learning Environment [Bellemare et al., 2013].