mlsanityData to 14 Oct 2026

Meta-Learning to Explore via Memory Density Feedback

arXiv 2503.02831 ↗First posted 4 Mar 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Exploration algorithms for reinforcement learning typically replace or augment the reward function with an additional ``intrinsic'' reward that trains the agent to seek previously unseen states of the environment. Here, we consider an exploration algorithm that exploits meta-learning, or learning to learn, such that the agent learns to maximize its exploration progress within a single episode, even between epochs of training. The agent learns a policy that aims to minimize the probability density of new observations with respect to all of its memories. In addition, it receives as feedback evaluations of the current observation density and retains that feedback in a recurrent network. By remembering trajectories of density, the agent learns to navigate a complex and growing landscape of familiarity in real-time, allowing it to maximize its exploration progress even in completely novel states of the environment for which its policy has not been trained.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 25 references that are in the map. First posted on arXiv 4 Mar 2025.

Builds on

13 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Reservoir Computing for Fast, Simplified Reinforcement Learning on Memory Tasks17 Dec

Other references · 12

  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2018Exploration by Random Network Distillationcited by 993
  5. 2016Learning to reinforcement learncited by 377
  6. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  7. 2020First return, then explorecited by 266
  8. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  9. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  10. 2023Unlocking the Power of Representations in Long-term Novelty-based Explorationcited by 9
  11. 2022Cell-Free Latent Go-Explorecited by 5
  12. 2019Reservoirs learn to learncited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploitation Is All You Need... for Exploration
  2. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Reward
  3. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?
  4. 2026Meta-Reinforcement Learning with Self-Reflection for Agentic Search
  5. 2025Memento 2: Learning by Stateful Reflective Memory
  6. 2026Unsupervised Learning of Efficient Exploration: Pre-training Adaptive Policies via Self-Imposed Goals
  7. 2024Memory Sequence Length of Data Sampling Impacts the Adaptation of Meta-Reinforcement Learning Agents
  8. 2019Learning Exploration Policies for Navigation