# Been There, Done That: Meta-Learning with Episodic Recall

arXiv: https://arxiv.org/abs/1805.09692 · HTML: https://mlsanity.com/paper/1805.09692/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2018.

## Built on by (most cited first, 26 of 26)

- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [Transformers are Meta-Reinforcement Learners](https://mlsanity.com/paper/2206.06614/) (2022)
- [Learning where to learn: Gradient sparsity in meta and continual learning](https://mlsanity.com/paper/2110.14402/) (2021)
- [Synthetic Returns for Long-Term Credit Assignment](https://mlsanity.com/paper/2102.12425/) (2021)
- [Large-Scale Retrieval for Reinforcement Learning](https://mlsanity.com/paper/2206.05314/) (2022)
- [How to Train Your MAML to Excel in Few-Shot Classification](https://mlsanity.com/paper/2106.16245/) (2021)
- [Curious Replay for Model-based Adaptation](https://mlsanity.com/paper/2306.15934/) (2023)
- [HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving](https://mlsanity.com/paper/2102.11344/) (2021)
- [Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences](https://mlsanity.com/paper/2509.16189/) (2025)
- [Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments](https://mlsanity.com/paper/2504.19139/) (2025)
- [Meta-Learning Strategies through Value Maximization in Neural Networks](https://mlsanity.com/paper/2310.19919/) (2023)
- [Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management](https://mlsanity.com/paper/2512.21567/) (2025)
- [Meta-Learning in Spiking Neural Networks with Reward-Modulated STDP](https://mlsanity.com/paper/2306.04410/) (2023)
- [Grid Cell-Inspired Fragmentation and Recall for Efficient Map Building](https://mlsanity.com/paper/2307.05793/) (2023)
- [A Miniature Brain Transformer: Thalamic Gating, Hippocampal Lateralization, Amygdaloid Salience, and Prefrontal Working Memory in Attention-Coupled Latent Memory](https://mlsanity.com/paper/2603.07217/) (2026)
- [Multi-task Batch Reinforcement Learning with Metric Learning](https://mlsanity.com/paper/1909.11373/) (2019)
- [Graceful Forgetting II. Data as a Process](https://mlsanity.com/paper/2211.15441/) (2022)
- [Agentic Episodic Control](https://mlsanity.com/paper/2506.01442/) (2025)
- [From Memories to Maps: Mechanisms of In-Context Reinforcement Learning in Transformers](https://mlsanity.com/paper/2506.19686/) (2025)
- [The Emergence of Abstract and Episodic Neurons in Episodic Meta-RL](https://mlsanity.com/paper/2104.02959/) (2021)
- [An introduction to reinforcement learning for neuroscience](https://mlsanity.com/paper/2311.07315/) (2023)
- [Reconciling Shared versus Context-Specific Information in a Neural Network Model of Latent Causes](https://mlsanity.com/paper/2312.08519/) (2023)
- [Evolving Afferent Architectures: Biologically-inspired Models for Damage-Avoidance Learning](https://mlsanity.com/paper/2602.04807/) (2026)
- [CDRL: A Reinforcement Learning Framework Inspired by Cerebellar Circuits and Dendritic Computational Strategies](https://mlsanity.com/paper/2602.15367/) (2026)
- [A neural network that maintains and retrieves memories based on context](https://mlsanity.com/paper/2609.37791/) (2026)
- [Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modeling](https://mlsanity.com/paper/2610.14913/) (2026)

## Abstract (arXiv)

Meta-learning agents excel at rapidly learning new tasks from open-ended task distributions; yet, they forget what they learn about each task as soon as the next begins. When tasks reoccur - as they do in natural environments - metalearning agents must explore again instead of immediately exploiting previously discovered solutions. We propose a formalism for generating open-ended yet repetitious environments, then develop a meta-learning architecture for solving these environments. This architecture melds the standard LSTM working memory with a differentiable neural episodic memory. We explore the capabilities of agents with this episodic LSTM in five meta-learning environments with reoccurring tasks, ranging from bandits to navigation and stochastic sequential decision problems.