mlsanityData to 14 Oct 2026

Recurrent Hypernetworks are Surprisingly Strong in Meta-RL

arXiv 2309.14970 ↗First posted 26 Sep 2023cs.LG cs.AI cs.ROcited by 16

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (RL) is notoriously impractical to deploy due to sample inefficiency. Meta-RL directly addresses this sample inefficiency by learning to perform few-shot learning when a distribution of related tasks is available for meta-training. While many specialized meta-RL methods have been proposed, recent work suggests that end-to-end learning in conjunction with an off-the-shelf sequential model, such as a recurrent network, is a surprisingly strong baseline. However, such claims have been controversial due to limited supporting evidence, particularly in the face of prior work establishing precisely the opposite. In this paper, we conduct an empirical investigation. While we likewise find that a recurrent network can achieve strong performance, we demonstrate that the use of hypernetworks is crucial to maximizing their potential. Surprisingly, when combined with hypernetworks, the recurrent baselines that are far simpler than existing specialized methods actually achieve the strongest performance of all methods evaluated. We provide code at https://github.com/jacooba/hyper.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 24 of its 31 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

24 of 31 references.

References · 24

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  3. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  4. 2016HyperNetworkscited by 1,027
  5. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  6. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  7. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  8. 2016Learning to reinforcement learncited by 377
  9. 2017Meta Networkscited by 269
  10. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  11. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  12. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
12 more
  1. 2023A Tutorial on Meta-Reinforcement Learningcited by 108
  2. 2019Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulationcited by 103
  3. 2019Meta reinforcement learning as task inferencecited by 77
  4. 2020Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescited by 59
  5. 2022Hypernetworks in Meta-Reinforcement Learningcited by 42
  6. 2019Generalization of Reinforcement Learners with Working and Episodic Memorycited by 36
  7. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 35
  8. 2020Rapid Task-Solving in Novel Environmentscited by 23
  9. 2021HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworkscited by 18
  10. 2022HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworkscited by 17
  11. 2020Learning Adaptive Exploration Strategies in Dynamic Environments Through Informed Policy Regularizationcited by 16
  12. 2021Linear Representation Meta-Reinforcement Learning for Instant Adaptationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 4; 2026: 4.

16 papers.

Sort:

By area:

  1. 2024XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learningcited by 14
  2. 2024Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Controlcited by 11
  3. 2024HyperMARL: Adaptive Hypernetworks for Multi-Agent RLcited by 11
  4. 2025HyperVLA: Efficient Inference in Vision-Language-Action Models via Hypernetworkscited by 7
  5. 2024DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learningcited by 4
  6. 2024Metalic: Meta-Learning In-Context with Protein Language Modelscited by 3
  7. 2025Capability-Aware Shared Hypernetworks for Flexible Heterogeneous Multi-Robot Coordinationcited by 3
  8. 2024SplAgger: Split Aggregation for Meta-Reinforcement Learningcited by 2
  9. 2024Enhancing Diversity in Bayesian Deep Learning via Hyperspherical Energy Minimization of CKAcited by 2
  10. 2024Entropy Regularized Task Representation Learning for Offline Meta-Reinforcement Learningcited by 2
  11. 2025TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environmentscited by 1
  12. 2025Hypernetworks for Model-Heterogeneous Personalized Federated Learningcited by 1
4 more
  1. 2026Hypernetworks for Dynamic Feature Selectioncited by 0
  2. 2026MATE: Solving Contextual Markov Decision Processes with Memory of Accumulated Transition Embeddingscited by 0
  3. 2026Knowledge Reutilization in Meta-Reinforcement Learningcited by 0
  4. 2026MOCHA: Multi-Objective Co-Design using Hypernetwork Architecturescited by 0