Data to 7 Oct 2026

Assessing Generalization in Deep Reinforcement Learning

arXiv 1810.12282 ↗First posted 29 Oct 2018cs.LG cs.AI stat.MLcited by 132

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (RL) has achieved breakthrough results on many tasks, but agents often fail to generalize beyond the environment they were trained in. As a result, deep RL algorithms that promote generalization are receiving increasing attention. However, works in this area use a wide variety of tasks and experimental setups for evaluation. The literature lacks a controlled assessment of the merits of different generalization schemes. Our aim is to catalyze community-wide progress on generalization in deep RL. To this end, we present a benchmark and experimental protocol, and conduct a systematic empirical study. Our framework contains a diverse set of environments, our methodology covers both in-distribution and out-of-distribution generalization, and our evaluation includes deep RL algorithms that specifically tackle generalization. Our key finding is that `vanilla' deep RL algorithms generalize better than specialized schemes that were proposed specifically to tackle generalization.

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 31; 2023: 32; 2024: 15; 2025: 21; 2026: 15.

132 papers.

Sort:

By area:

2022Evolving Curricula with Regret-Based Environment Designcited by 166
2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 78
2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
2023What Makes Pre-Trained Visual Representations Successful for Robust Manipulation?cited by 38
2021Adversarial Reinforcement Learning for Procedural Content Generationcited by 36
2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36

38 more

2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 31
2022When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learningcited by 31
2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
2023The Generalization Gap in Offline Reinforcement Learningcited by 25
2024Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximationcited by 25
2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
2022Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learningcited by 21
2023Towards Minimax Optimality of Model-based Robust Reinforcement Learningcited by 21
2021When Is Generalizable Reinforcement Learning Tractable?cited by 19
2022Efficient Policy Iteration for Robust Markov Decision Processes via Regularizationcited by 19
2022Online Policy Optimization for Robust MDPcited by 19
2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
2021Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimizationcited by 18
2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 18
2024Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learningcited by 17
2024Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detectioncited by 17
2022A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learningcited by 15
2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RLcited by 15
2025Horizon Generalization in Reinforcement Learningcited by 15
2021Learning a subspace of policies for online adaptation in Reinforcement Learningcited by 14
2023Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregressioncited by 14
2021Correcting Momentum in Temporal Difference Learningcited by 12
2021Learning Domain Invariant Representations in Goal-conditioned Block MDPscited by 12
2022Learning General World Models in a Handful of Reward-Free Deploymentscited by 12
2023Verifying Generalization in Deep Learningcited by 11
2024DRED: Zero-Shot Transfer in Reinforcement Learning via Data-Regularised Environment Designcited by 11
2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordiacited by 11
2021A Survey on Deep Reinforcement Learning for Data Processing and Analyticscited by 10
2022Local Feature Swapping for Generalization in Reinforcement Learningcited by 9
2022Generalization, Mayhems and Limits in Recurrent Proximal Policy Optimizationcited by 9
2023Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectoriescited by 9
2024A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedbackcited by 9
2021Joint Attention for Multi-Agent Coordination and Social Learningcited by 8
2021Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRLcited by 8
2021Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RLcited by 8
2022On the Generalization and Adaption Performance of Causal Modelscited by 8

50 most cited of 132 All 132 (JSON)