mlsanityData to 7 Oct 2026

A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning

arXiv 1806.07937 ↗First posted 20 Jun 2018cs.LG cs.AI stat.MLcited by 86

Not on the map. Why?

Abstract (arXiv)

The risks and perils of overfitting in machine learning are well known. However most of the treatment of this, including diagnostic tools and remedies, was developed for the supervised learning case. In this work, we aim to offer new perspectives on the characterization and prevention of overfitting in deep Reinforcement Learning (RL) methods, with a particular focus on continuous domains. We examine several aspects, such as how to define and diagnose overfitting in MDPs, and how to reduce risks by injecting sufficient training diversity. This work complements recent findings on the brittleness of deep RL methods and offers practical observations for RL researchers and practitioners.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 20; 2022: 16; 2023: 23; 2024: 12; 2025: 10; 2026: 4.

86 papers.

Sort:

By area:

  1. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  2. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  3. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  5. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  6. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  7. 2021Replay-Guided Adversarial Environment Designcited by 108
  8. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
  9. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  10. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
  11. 2023XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAXcited by 54
  12. 2022Exploration via Elliptical Episodic Bonusescited by 51
38 more
  1. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  2. 2021Adversarially Guided Actor-Criticcited by 47
  3. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
  4. 2022Melting Pot 2.0cited by 37
  5. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
  6. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
  7. 2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 31
  8. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  9. 2026Maximum Likelihood Reinforcement Learningcited by 29
  10. 2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
  11. 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 26
  12. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
  13. 2025Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Rewardcited by 25
  14. 2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
  15. 2023In-Context Reinforcement Learning for Variable Action Spacescited by 23
  16. 2023Causal Reinforcement Learning: A Surveycited by 22
  17. 2021When Is Generalizable Reinforcement Learning Tractable?cited by 19
  18. 2021Towards Deeper Deep Reinforcement Learning with Spectral Normalizationcited by 19
  19. 2022CLUTR: Curriculum Learning via Unsupervised Task Representation Learningcited by 19
  20. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
  21. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  22. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 18
  23. 2022Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worldscited by 17
  24. 2023Stabilizing Unsupervised Environment Design with a Learned Adversarycited by 17
  25. 2021Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentationcited by 15
  26. 2023Generalizable Visual Reinforcement Learning with Segment Anything Modelcited by 12
  27. 2022Learn the Time to Learn: Replay Scheduling in Continual Learningcited by 11
  28. 2024The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Designcited by 11
  29. 2023Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RLcited by 10
  30. 2022Local Feature Swapping for Generalization in Reinforcement Learningcited by 9
  31. 2023Generalization through Diversity: Improving Unsupervised Environment Designcited by 9
  32. 2023Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectoriescited by 9
  33. 2023Mind the Model, Not the Agent: The Primacy Bias in Model-based RLcited by 9
  34. 2024Adversarial Environment Design via Regret-Guided Diffusion Modelscited by 9
  35. 2021Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RLcited by 8
  36. 2025On Generalization Across Environments In Multi-Objective Reinforcement Learningcited by 8
  37. 2024Investigating Generalization Behaviours of Generative Flow Networkscited by 6
  38. 2024The Perils of Optimizing Learned Reward Functions: Low Training Error Does Not Guarantee Low Regretcited by 6

50 most cited of 86 All 86 (JSON)