mlsanityData to 14 Oct 2026

Episodic Memory Deep Q-Networks

arXiv 1805.07603 ↗First posted 19 May 2018cs.LG cs.AI stat.MLcited by 42

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) algorithms have made huge progress in recent years by leveraging the power of deep neural networks (DNN). Despite the success, deep RL algorithms are known to be sample inefficient, often requiring many rounds of interaction with the environments to obtain satisfactory performance. Recently, episodic memory based RL has attracted attention due to its ability to latch on good actions quickly. In this paper, we present a simple yet effective biologically inspired RL algorithm called Episodic Memory Deep Q-Networks (EMDQN), which leverages episodic memory to supervise an agent during training. Experiments show that our proposed method can lead to better sample efficiency and is more likely to find good policies. It only requires 1/5 of the interactions of DQN to achieve many state-of-the-art performances on Atari games, significantly outperforming regular DQN and other episodic memory based RL algorithms.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 11; 2022: 6; 2023: 6; 2024: 8; 2025: 4; 2026: 6.

42 papers.

Sort:

By area:

  1. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  2. 2021Offline Reinforcement Learning with Value-based Episodic Memorycited by 36
  3. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 36
  4. 2022Large-Scale Retrieval for Reinforcement Learningcited by 24
  5. 2021Generalizable Episodic Memory for Deep Reinforcement Learningcited by 23
  6. 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisitioncited by 21
  7. 2020DeepAveragers: Offline Reinforcement Learning by Solving Derived Non-Parametric MDPscited by 16
  8. 2021Model-Based Episodic Memory Induces Dynamic Hybrid Controlscited by 14
  9. 2021Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximationcited by 11
  10. 2026UI-Voyager: A Self-Evolving GUI Agent Learning via Failed Experiencecited by 11
  11. 2022A Machine with Short-Term, Episodic, and Semantic Memory Systemscited by 9
  12. 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
30 more
  1. 2021Solving Continuous Control with Episodic Memorycited by 8
  2. 2021Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples with On-Policy Experiencecited by 8
  3. 2023Neural Episodic Control with State Abstractioncited by 7
  4. 2024MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Tradingcited by 7
  5. 2024Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learningcited by 5
  6. 2026MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGscited by 5
  7. 2021On the Estimation Bias in Double Q-Learningcited by 4
  8. 2026Memory poisoning and secure multi-agent systemscited by 4
  9. 2021Successor Feature Neural Episodic Controlcited by 3
  10. 2021Sequential memory improves sample and memory efficiency in Episodic Controlcited by 3
  11. 2023Sample Dropout: A Simple yet Effective Variance Reduction Technique in Deep Policy Optimizationcited by 3
  12. 2024Episodic Reinforcement Learning with Expanded State-reward Spacecited by 3
  13. 2024Learning Diverse Policies with Soft Self-Generated Guidancecited by 3
  14. 2024Enhancing Q-Learning with Large Language Model Heuristicscited by 2
  15. 2025RANa: Retrieval-Augmented Navigationcited by 2
  16. 2021State-based Episodic Memory for Multi-Agent Reinforcement Learningcited by 1
  17. 2022Continuous Episodic Controlcited by 1
  18. 2023Two-Memory Reinforcement Learningcited by 1
  19. 2023Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularizationcited by 1
  20. 2024HG2P: Hippocampus-inspired High-reward Graph and Model-Free Q-Gradient Penalty for Path Planning and Motion Controlcited by 1
  21. 2025Agentic Episodic Controlcited by 1
  22. 2022Soft Actor-Critic with Inhibitory Networks for Faster Retrainingcited by 0
  23. 2022Eigen Memory Treescited by 0
  24. 2022Knowing the Past to Predict the Future: Reinforcement Virtual Learningcited by 0
  25. 2023Deep Reinforcement Learning with Task-Adaptive Retrieval via Hypernetworkcited by 0
  26. 2024Highway Graph to Accelerate Reinforcement Learningcited by 0
  27. 2024High-fidelity social learning via shared episodic memories enhances collaborative foraging through mnemonic convergencecited by 0
  28. 2026Learning From Failures: Efficient Reinforcement Learning Control with Episodic Memorycited by 0
  29. 2026Artifacts as Memory Beyond the Agent Boundarycited by 0
  30. 2026Episodic Memory Temporal Consistency for Cooperative Multi-Agent Reinforcement Learningcited by 0