mlsanityData to 14 Oct 2026

Dynamic Memory-based Curiosity: A Bootstrap Approach for Exploration

arXiv 2208.11349 ↗First posted 24 Aug 2022cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The sparsity of extrinsic rewards poses a serious challenge for reinforcement learning (RL). Currently, many efforts have been made on curiosity which can provide a representative intrinsic reward for effective exploration. However, the challenge is still far from being solved. In this paper, we present a novel curiosity for RL, named DyMeCu, which stands for Dynamic Memory-based Curiosity. Inspired by human curiosity and information theory, DyMeCu consists of a dynamic memory and dual online learners. The curiosity arouses if memorized information can not deal with the current state, and the information gap between dual learners can be formulated as the intrinsic reward for agents, and then such state information can be consolidated into the dynamic memory. Compared with previous curiosity methods, DyMeCu can better mimic human curiosity with dynamic memory, and the memory module can be dynamically grown based on a bootstrap paradigm with dual learners. On multiple benchmarks including DeepMind Control Suite and Atari Suite, large-scale empirical experiments are conducted and the results demonstrate that DyMeCu outperforms competitive curiosity-based methods with or without extrinsic rewards. We will release the code to enhance reproducibility.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 55 references that are in the map. First posted on arXiv 24 Aug 2022.

Builds on

31 of 55 references.

References · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  7. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  8. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  9. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  10. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  11. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  12. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
19 more
  1. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  2. 2019Self-Supervised Exploration via Disagreementcited by 265
  3. 2017Automated Curriculum Learning for Neural Networkscited by 257
  4. 2019Efficient Exploration via State Marginal Matchingcited by 188
  5. 2021Reinforcement Learning with Prototypical Representationscited by 179
  6. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  7. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
  8. 2021APS: Active Pretraining with Successor Featurescited by 112
  9. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 112
  10. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 102
  11. 2017Surprise-Based Intrinsic Motivation for Deep Reinforcement Learningcited by 83
  12. 2021Replay in Deep Learning: Current Approaches and Missing Biological Elementscited by 72
  13. 2021Bootstrapped Meta-Learningcited by 45
  14. 2020Active World Model Learning with Progress Curiositycited by 28
  15. 2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 27
  16. 2020Novelty Search in Representational Space for Sample Efficient Explorationcited by 26
  17. 2018Curiosity-driven reinforcement learning with homeostatic regulationcited by 4
  18. 2019Visual novelty, curiosity, and intrinsic reward in machine learning and the braincited by 4
  19. 2016Infomax strategies for an optimal balance between exploration and exploitationcited by 3

Built on by

Not cited yet.