mlsanityData to 14 Oct 2026

Auxiliary task discovery through generate-and-test

arXiv 2210.14361 ↗First posted 25 Oct 2022cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we explore an approach to auxiliary task discovery in reinforcement learning based on ideas from representation learning. Auxiliary tasks tend to improve data efficiency by forcing the agent to learn auxiliary prediction and control objectives in addition to the main task of maximizing reward, and thus producing better representations. Typically these tasks are designed by people. Meta-learning offers a promising avenue for automatic task discovery; however, these methods are computationally expensive and challenging to tune in practice. In this paper, we explore a complementary approach to the auxiliary task discovery: continually generating new auxiliary tasks and preserving only those with high utility. We also introduce a new measure of auxiliary tasks' usefulness based on how useful the features induced by them are for the main task. Our discovery algorithm significantly outperforms random tasks and learning without auxiliary tasks across a suite of environments.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

24 of 33 references.

References · 24

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  5. 2018How to train your MAMLcited by 309
  6. 2018Stronger generalization bounds for deep nets via a compression approachcited by 307
  7. 2016Learning to Navigate in Complex Environmentscited by 198
  8. 2019Meta-Learning Representations for Continual Learningcited by 189
  9. 2018Meta-Gradient Reinforcement Learningcited by 159
  10. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  11. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 100
  12. 2016The Predictron: End-To-End Learning and Planningcited by 93
12 more
  1. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 80
  2. 2020Discovering Reinforcement Learning Algorithmscited by 77
  3. 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
  4. 2020The Value-Improvement Path: Towards Better Representations for Reinforcement Learningcited by 60
  5. 2021On The Effect of Auxiliary Tasks on Representation Dynamicscited by 57
  6. 2020Meta-Gradient Reinforcement Learning with an Objective Discovered Onlinecited by 52
  7. 2019Discovery of Useful Questions as Auxiliary Taskscited by 45
  8. 2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
  9. 2022Reward-Respecting Subtasks for Model-Based Reinforcement Learningcited by 16
  10. 2022Goal-Space Planning with Subgoal Modelscited by 12
  11. 2021Learning State Representations from Random Deep Action-conditional Predictionscited by 6
  12. 2021Scalable Online Recurrent Learning Using Columnar Neural Networkscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Test-Time Warmup for Multimodal Large Language Modelscited by 0
  2. 2025Generating Auxiliary Tasks with Reinforcement Learningcited by 0