mlsanityData to 14 Oct 2026

Dueling Network Architectures for Deep Reinforcement Learning

arXiv 1511.06581 ↗First posted 20 Nov 2015cs.LGcited by 785

Not on the map. Why?

Cited by 190 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In recent years there have been many successes of using deep representations in reinforcement learning. Still, many of these applications use conventional architectures, such as convolutional networks, LSTMs, or auto-encoders. In this paper, we present a new neural network architecture for model-free reinforcement learning. Our dueling network represents two separate estimators: one for the state value function and one for the state-dependent action advantage function. The main benefit of this factoring is to generalize learning across actions without imposing any change to the underlying reinforcement learning algorithm. Our results show that this architecture leads to better policy evaluation in the presence of many similar-valued actions. Moreover, the dueling architecture enables our RL agent to outperform the state-of-the-art on the Atari 2600 domain.

Key facts. Cited by 785 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 3; 2019: 3; 2020: 2; 2021: 191; 2022: 150; 2023: 124; 2024: 127; 2025: 122; 2026: 63.

785 papers.

Sort:

By area:

  1. 2022Transformers are Sample-Efficient World Modelscited by 328
  2. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
  3. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  4. 2021Mastering Atari Games with Limited Datacited by 237
  5. 2022Online Decision Transformercited by 196
  6. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 187
  7. 2023Understanding plasticity in neural networkscited by 174
  8. 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 161
  9. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 147
  10. 2018Self-Imitation Learningcited by 138
  11. 2021The Benchmark Lotterycited by 136
  12. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
38 more
  1. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  2. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  3. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  4. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 97
  5. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
  6. 2024Simplifying Deep Temporal Difference Learningcited by 78
  7. 2021Off-Belief Learningcited by 75
  8. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
  9. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 73
  10. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  11. 2023Heterogeneous-Agent Reinforcement Learningcited by 69
  12. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  13. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 65
  14. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 65
  15. 2023Generative Flow Networks as Entropy-Regularized RLcited by 64
  16. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 61
  17. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 59
  18. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  19. 2021Learning to run a Power Network Challenge: a Retrospective Analysiscited by 55
  20. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 50
  21. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  22. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 50
  23. 2024Value Augmented Sampling for Language Model Alignment and Personalizationcited by 49
  24. 2021Constructions in combinatorics via neural networkscited by 48
  25. 2023A Survey on Causal Reinforcement Learningcited by 45
  26. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 44
  27. 2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 43
  28. 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 43
  29. 2023PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learningcited by 42
  30. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 42
  31. 2021The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 41
  32. 2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
  33. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 41
  34. 2022Mask-based Latent Reconstruction for Reinforcement Learningcited by 39
  35. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 39
  36. 2019An Optimistic Perspective on Offline Reinforcement Learningcited by 37
  37. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  38. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 35

50 most cited of 785 All 785 (JSON)