mlsanityData to 7 Oct 2026

Hyperparameters in Contextual RL are Highly Situational

arXiv 2212.10876 ↗First posted 21 Dec 2022cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Although Reinforcement Learning (RL) has shown impressive results in games and simulation, real-world application of RL suffers from its instability under changing environment conditions and hyperparameters. We give a first impression of the extent of this instability by showing that the hyperparameters found by automatic hyperparameter optimization (HPO) methods are not only dependent on the problem at hand, but even on how well the state describes the environment dynamics. Specifically, we show that agents in contextual RL require different hyperparameters if they are shown how environmental factors change. In addition, finding adequate hyperparameter configurations is not equally easy for both settings, further highlighting the need for research into how hyperparameters influence learning and generalization in RL.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 23 of its 35 references that are in the map. First posted on arXiv 21 Dec 2022.

Builds on

23 of 35 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2016OpenAI Gymcited by 1,684
  5. 2017Deep Reinforcement Learning that Matterscited by 727
  6. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  7. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  8. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  9. 2016Learning to reinforcement learncited by 369
  10. 2017Population Based Training of Neural Networkscited by 351
  11. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  12. 2015Contextual Markov Decision Processescited by 213
11 more
  1. 2017Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Controlcited by 92
  2. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  3. 2020Provably Efficient Online Hyperparameter Optimization with Population-Based Banditscited by 61
  4. 2017Markov Decision Processes with Continuous Side Informationcited by 54
  5. 2021Evolving Reinforcement Learning Algorithmscited by 48
  6. 2020Sample-Efficient Automated Deep Reinforcement Learningcited by 31
  7. 2018Learning to Design RNAcited by 27
  8. 2019A Cooperative Multi-Agent Reinforcement Learning Framework for Resource Balancing in Complex Logistics Networkcited by 22
  9. 2020TOAD-GAN: Coherent Style Level Generation from a Single Examplecited by 22
  10. 2021CARL: A Benchmark for Contextual and Adaptive Reinforcement Learningcited by 21
  11. 2020Quantity vs. Quality: On Hyperparameter Optimization for Deep Reinforcement Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 2.

4 papers.

  1. 2024A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learningcited by 15
  2. 2026Contextual Intelligence The Next Leap for Reinforcement Learningcited by 1
  3. 2023Optimizing Heat Alert Issuance with Reinforcement Learningcited by 0
  4. 2026Dynamic Context Scheduling: Learning Beyond the Static Universecited by 0