mlsanityData to 7 Oct 2026

Model-Based Episodic Memory Induces Dynamic Hybrid Controls

arXiv 2111.02104 ↗First posted 3 Nov 2021cs.LG cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Episodic control enables sample efficiency in reinforcement learning by recalling past experiences from an episodic memory. We propose a new model-based episodic memory of trajectories addressing current limitations of episodic control. Our memory estimates trajectory values, guiding the agent towards good policies. Built upon the memory, we construct a complementary learning model via a dynamic hybrid control unifying model-based, episodic and habitual learning into a single architecture. Experiments demonstrate that our model allows significantly faster and better learning than other strong reinforcement learning agents across a variety of environments including stochastic and non-Markovian settings.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 24 of its 50 references that are in the map. First posted on arXiv 3 Nov 2021.

Builds on

24 of 50 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  4. 2015Prioritized Experience Replaycited by 1,070
  5. 2020Mastering Atari with Discrete World Modelscited by 990
  6. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  7. 2014Neural Turing Machinescited by 630
  8. 2019Model-Based Reinforcement Learning for Ataricited by 495
  9. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  10. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 162
  11. 2017Neural Episodic Controlcited by 148
  12. 2016Model-Free Episodic Controlcited by 100
12 more
  1. 2018Optimizing Agent Behavior over Long Time Scales by Transporting Valuecited by 57
  2. 2018Unsupervised Predictive Memory in a Goal-Directed Agentcited by 54
  3. 2018Episodic Memory Deep Q-Networkscited by 41
  4. 2019Generalization of Reinforcement Learners with Working and Episodic Memorycited by 35
  5. 2020Self-Attentive Associative Memorycited by 33
  6. 2018Sample-Efficient Deep Reinforcement Learning via Episodic Backward Updatecited by 29
  7. 2021Generalizable Episodic Memory for Deep Reinforcement Learningcited by 22
  8. 2016Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tighteningcited by 21
  9. 2018Fast deep reinforcement learning using online adjustments from the pastcited by 14
  10. 2019Learning to Remember More with Less Memorizationcited by 14
  11. 2020Neurocoder: Learning General-Purpose Computation Using Stored Neural Programscited by 6
  12. 2018Faster Deep Q-learning using Neural Episodic Controlcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 4; 2024: 2; 2025: 4; 2026: 1.

14 papers.

Sort:

By area:

  1. 2025Episodic memory in AI agents poses risks that should be studied and mitigatedcited by 11
  2. 2024Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learningcited by 8
  3. 2023Neural Episodic Control with State Abstractioncited by 7
  4. 2023Memory-Augmented Theory of Mind Networkcited by 6
  5. 2025Reasoning Under 1 Billion: Memory-Augmented Reinforcement Learning for Large Language Modelscited by 6
  6. 2021Episodic Policy Gradient Trainingcited by 5
  7. 2023Beyond Surprise: Improving Exploration Through Surprise Noveltycited by 5
  8. 2022Learning to Constrain Policy Optimization with Virtual Trust Regioncited by 4
  9. 2024Large Language Models Prompting With Episodic Memorycited by 4
  10. 2025Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigationcited by 3
  11. 2025Patch-based Memory Gate Model in Time Series Foundation Modelcited by 2
  12. 2022Continuous Episodic Controlcited by 1
2 more
  1. 2026Continual Fine-Tuning of Large Language Models via Program Memorycited by 1
  2. 2023Policy Optimization with Smooth Guidance Learned from State-Only Demonstrationscited by 0