Data to 7 Oct 2026

Generalization and Regularization in DQN

arXiv 1810.00123 ↗First posted 29 Sep 2018cs.LG cs.AI stat.MLcited by 117

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning algorithms have shown an impressive ability to learn complex control policies in high-dimensional tasks. However, despite the ever-increasing performance on popular benchmarks, policies learned by deep reinforcement learning algorithms can struggle to generalize when evaluated in remarkably similar environments. In this paper we propose a protocol to evaluate generalization in reinforcement learning through different modes of Atari 2600 games. With that protocol we assess the generalization capabilities of DQN, one of the most traditional deep reinforcement learning algorithms, and we provide evidence suggesting that DQN overspecializes to the training environment. We then comprehensively evaluate the impact of dropout and ℓ2 regularization, as well as the impact of reusing learned representations to improve the generalization capabilities of DQN. Despite regularization being largely underutilized in deep reinforcement learning, we show that it can, in fact, help DQN learn more general features. These features can be reused and fine-tuned on similar tasks, considerably improving DQN's sample efficiency.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 28; 2023: 19; 2024: 25; 2025: 13; 2026: 4.

117 papers.

Sort:

By area:

2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
2021Domain Generalization with MixStylecited by 563
2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
2024Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithmscited by 139
2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 118
2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97

38 more

2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizescited by 72
2021MixStyle Neural Networks for Domain Generalization and Adaptationcited by 64
2021Adversarially Guided Actor-Criticcited by 47
2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
2023What Makes Pre-Trained Visual Representations Successful for Robust Manipulation?cited by 38
2022Melting Pot 2.0cited by 37
2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 36
2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 35
2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
2021DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneckcited by 31
2022Chain of Thought Imitation with Procedure Cloningcited by 30
2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
2021Robust Predictable Controlcited by 29
2023The Generalization Gap in Offline Reinforcement Learningcited by 25
2024Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximationcited by 25
2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
2021Adaptive Rational Activations to Boost Deep Reinforcement Learningcited by 23
2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
2021Return-based Scaling: Yet Another Normalisation Trick for Deep RLcited by 22
2022Mastering the Unsupervised Reinforcement Learning Benchmark from Pixelscited by 20
2023Explore to Generalize in Zero-Shot RLcited by 20
2021When Is Generalizable Reinforcement Learning Tractable?cited by 19
2021Towards Deeper Deep Reinforcement Learning with Spectral Normalizationcited by 19
2021Sequoia: A Software Framework to Unify Continual Learning Researchcited by 19
2022Online Policy Optimization for Robust MDPcited by 19
2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
2023Gradient-based Planning with World Modelscited by 18
2022Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worldscited by 17
2024Dissecting Deep RL with High Update Ratios: Combatting Value Divergencecited by 17
2024Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learningcited by 17
2022Regularized Q-learningcited by 16
2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
2024Upper and Lower Bounds for Distributionally Robust Off-Dynamics Reinforcement Learningcited by 15
2025Horizon Generalization in Reinforcement Learningcited by 15
2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learningcited by 14
2023Normalization Enhances Generalization in Visual Reinforcement Learningcited by 13

50 most cited of 117 All 117 (JSON)