mlsanityData to 7 Oct 2026

The Role of Exploration for Task Transfer in Reinforcement Learning

arXiv 2210.06168 ↗First posted 11 Oct 2022cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

The exploration--exploitation trade-off in reinforcement learning (RL) is a well-known and much-studied problem that balances greedy action selection with novel experience, and the study of exploration methods is usually only considered in the context of learning the optimal policy for a single learning task. However, in the context of online task transfer, where there is a change to the task during online operation, we hypothesize that exploration strategies that anticipate the need to adapt to future tasks can have a pronounced impact on the efficiency of transfer. As such, we re-examine the exploration--exploitation trade-off in the context of transfer learning. In this work, we review reinforcement learning exploration methods, define a taxonomy with which to organize them, analyze these methods' differences in the context of task transfer, and suggest avenues for future investigation.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 53 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

27 of 53 references.

References · 27

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2020Knowledge Distillation: A Surveycited by 1,578
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  6. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  7. 2017Hindsight Experience Replaycited by 1,096
  8. 2018Exploration by Random Network Distillationcited by 966
  9. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  10. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  11. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  12. 2017A Tutorial on Thompson Samplingcited by 420
15 more
  1. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  2. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
  3. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  4. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  5. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  6. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  7. 2021Discovering and Achieving Goals via World Modelscited by 131
  8. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  9. 2022Deep Hierarchical Planning from Pixelscited by 96
  10. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  11. 2018Efficient Exploration through Bayesian Deep Q-Networkscited by 67
  12. 2015On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Modelscited by 52
  13. 2022NovGrid: A Flexible Grid World for Evaluating Agent Response to Noveltycited by 14
  14. 2015Online Transfer Learning in Reinforcement Learning Domainscited by 6
  15. 2019Addressing Sample Complexity in Visual Tasks Using HER and Hallucinatory GANscited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2.

2 papers.

  1. 2023Neuro-Symbolic World Models for Adapting to Open World Noveltycited by 14
  2. 2023Uncertainty-aware transfer across tasks using hybrid model-based successor feature reinforcement learningcited by 1