mlsanityData to 7 Oct 2026

Goal-Conditioned Generators of Deep Policies

arXiv 2207.01570 ↗First posted 4 Jul 2022cs.LG stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

Goal-conditioned Reinforcement Learning (RL) aims at learning optimal policies, given goals encoded in special command inputs. Here we study goal-conditioned neural nets (NNs) that learn to generate deep NN policies in form of context-specific weight matrices, similar to Fast Weight Programmers and other methods from the 1990s. Using context commands of the form "generate a policy that achieves a desired expected return," our NN generators combine powerful exploration of parameter space with generalization across commands to iteratively find better and better policies. A form of weight-sharing HyperNetworks and policy embeddings scales our method to generate deep NNs. Experiments show how a single learned policy generator can produce policies that achieve any return seen during training. Finally, we evaluate our algorithm on a set of continuous control tasks where it exhibits competitive performance. Our code is public.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 51 references that are in the map. First posted on arXiv 4 Jul 2022.

Builds on

24 of 51 references.

References · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2016HyperNetworkscited by 1,003
  7. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  8. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  9. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  10. 2019Continual learning with hypernetworkscited by 267
  11. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  12. 2019Training Agents using Upside-Down Reinforcement Learningcited by 106
12 more
  1. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  2. 2018Differentiable plasticity: training plastic neural networks with backpropagationcited by 83
  3. 2015On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Modelscited by 52
  4. 2020Meta Learning Backpropagation And Improving Itcited by 49
  5. 2020Meta-Learning through Hebbian Plasticity in Random Networkscited by 47
  6. 2022A Modern Self-Referential Weight Matrix That Learns to Modify Itselfcited by 40
  7. 2017Hindsight policy gradientscited by 36
  8. 2020Policy Evaluation Networkscited by 28
  9. 2021Introducing Symmetries to Black Box Meta Reinforcement Learningcited by 25
  10. 2020Parameter-Based Value Functionscited by 24
  11. 2022Upside-Down Reinforcement Learning Can Diverge in Stochastic Environments With Episodic Resetscited by 10
  12. 2020Finding online neural update rules by learning to remembercited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 1; 2025: 3; 2026: 1.

9 papers.

  1. 2022Hypernetworks for Zero-shot Transfer in Reinforcement Learningcited by 18
  2. 2024Learning Useful Representations of Recurrent Neural Network Weight Matricescited by 13
  3. 2023Learning to Identify Critical States for Reinforcement Learning from Videoscited by 9
  4. 2023Skill Decision Transformercited by 7
  5. 2025HyperVLA: Efficient Inference in Vision-Language-Action Models via Hypernetworkscited by 6
  6. 2022Learning One Abstract Bit at a Time Through Self-Invented Experiments Encoded as Neural Networkscited by 4
  7. 2025From Parameters to Behaviors: Unsupervised Compression of the Policy Spacecited by 4
  8. 2025Upside Down Reinforcement Learning with Policy Generatorscited by 1
  9. 2026Unsupervised Behavioral Compression: Learning Low-Dimensional Policy Manifolds through State-Occupancy Matchingcited by 1