mlsanityData to 14 Oct 2026

Hypernetworks for Zero-shot Transfer in Reinforcement Learning

arXiv 2211.15457 ↗First posted 28 Nov 2022cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

In this paper, hypernetworks are trained to generate behaviors across a range of unseen task conditions, via a novel TD-based training objective and data from a set of near-optimal RL solutions for training tasks. This work relates to meta RL, contextual RL, and transfer learning, with a particular focus on zero-shot performance at test time, enabled by knowledge of the task parameters (also known as context). Our technical approach is based upon viewing each RL algorithm as a mapping from the MDP specifics to the near-optimal value function and policy and seek to approximate it with a hypernetwork that can generate near-optimal value functions and policies, given the parameters of the MDP. We show that, under certain conditions, this mapping can be considered as a supervised learning problem. We empirically evaluate the effectiveness of our method for zero-shot transfer to new reward and transition dynamics on a series of continuous control tasks from DeepMind Control Suite. Our method demonstrates significant improvements over baselines from multitask and meta RL approaches.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 41 of its 67 references that are in the map. First posted on arXiv 28 Nov 2022.

Builds on

41 of 67 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2016Generative Adversarial Imitation Learningcited by 1,597
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  9. 2017Hindsight Experience Replaycited by 1,136
  10. 2016HyperNetworkscited by 1,027
  11. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  12. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
29 more
  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  2. 2018A Survey on Deep Transfer Learningcited by 419
  3. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  4. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 292
  5. 2019Continual learning with hypernetworkscited by 273
  6. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  7. 2015Contextual Markov Decision Processescited by 220
  8. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 214
  9. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  10. 2018Meta-Gradient Reinforcement Learningcited by 159
  11. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 125
  12. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  13. 2019Training Agents using Upside-Down Reinforcement Learningcited by 107
  14. 2018Universal Successor Features Approximatorscited by 103
  15. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 102
  16. 2019Hindsight Credit Assignmentcited by 79
  17. 2020learn2learn: A Library for Meta-Learning Researchcited by 69
  18. 2017Bayesian Hypernetworkscited by 66
  19. 2020Meta-Gradient Reinforcement Learning with an Objective Discovered Onlinecited by 52
  20. 2020Continual Model-Based Reinforcement Learning with Hypernetworkscited by 44
  21. 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
  22. 2017Hindsight policy gradientscited by 37
  23. 2019What Can Learned Intrinsic Rewards Capture?cited by 36
  24. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 35
  25. 2020Policy Evaluation Networkscited by 28
  26. 2020Universal Value Density Estimation for Imitation Learning and Goal-Conditioned Reinforcement Learningcited by 12
  27. 2022General Policy Evaluation and Improvement by Learning to Identify Few But Crucial Statescited by 10
  28. 2022Goal-Conditioned Generators of Deep Policiescited by 9
  29. 2022Operator Deep Q-Learning: Zero-Shot Reward Transferring in Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 4; 2026: 5.

19 papers.

Sort:

By area:

  1. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  2. 2023Universal Morphology Control via Contextual Modulationcited by 23
  3. 2024Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusioncited by 14
  4. 2024Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Controlcited by 11
  5. 2024Learning Pareto Set for Multi-Objective Continuous Robot Controlcited by 8
  6. 2024Model-Based Transfer Learning for Contextual Reinforcement Learningcited by 7
  7. 2025HyperVLA: Efficient Inference in Vision-Language-Action Models via Hypernetworkscited by 7
  8. 2025Hyper-GoalNet: Goal-Conditioned Manipulation Policy Learning with HyperNetworkscited by 4
  9. 2024Episodic Reinforcement Learning with Expanded State-reward Spacecited by 3
  10. 2024Learning to Look: Seeking Information for Decision Making via Policy Factorizationcited by 3
  11. 2025Capability-Aware Shared Hypernetworks for Flexible Heterogeneous Multi-Robot Coordinationcited by 3
  12. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shiftscited by 2
7 more
  1. 2023Domain Generalization Deep Graph Transformationcited by 1
  2. 2024TRAM: Test-Time Risk Adaptation with Mixture of Agentscited by 1
  3. 2026DCoPilot: Generative AI-Empowered Policy Adaptation for Dynamic Data Center Operationscited by 1
  4. 2026TeNet: Text-to-Network for Compact Policy Synthesiscited by 0
  5. 2026Temporal Consistency Improves Generalization in Contextual Offline Meta Reinforcement Learningcited by 0
  6. 2026Amortized Low-Rank Adaptation for Model-Based Reinforcement Learningcited by 0
  7. 2026MOCHA: Multi-Objective Co-Design using Hypernetwork Architecturescited by 0