mlsanityData to 7 Oct 2026

Neural Episodic Control with State Abstraction

arXiv 2301.11490 ↗First posted 27 Jan 2023cs.LG cs.AI cs.NEcited by 7

Not on the map. Why?

Abstract (arXiv)

Existing Deep Reinforcement Learning (DRL) algorithms suffer from sample inefficiency. Generally, episodic control-based approaches are solutions that leverage highly-rewarded past experiences to improve sample efficiency of DRL algorithms. However, previous episodic control-based approaches fail to utilize the latent information from the historical behaviors (e.g., state transitions, topological similarities, etc.) and lack scalability during DRL training. This work introduces Neural Episodic Control with State Abstraction (NECSA), a simple but effective state abstraction-based episodic control containing a more comprehensive episodic memory, a novel state evaluation, and a multi-step state analysis. We evaluate our approach to the MuJoCo and Atari tasks in OpenAI gym domains. The experimental results indicate that NECSA achieves higher sample efficiency than the state-of-the-art episodic control-based approaches. Our data and code are available at the project websitefootnoteurlhttps://sites.google.com/view/drl-necsa.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 65 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

32 of 65 references.

References · 32

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2020Mastering Atari with Discrete World Modelscited by 990
  6. 2018Exploration by Random Network Distillationcited by 966
  7. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  8. 2019Model-Based Reinforcement Learning for Ataricited by 495
  9. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  10. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  11. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  12. 2017Neural Episodic Controlcited by 148
20 more
  1. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 139
  2. 2016Model-Free Episodic Controlcited by 100
  3. 2019Discretizing Continuous Action Space for On-Policy Optimizationcited by 65
  4. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  5. 2018Episodic Memory Deep Q-Networkscited by 41
  6. 2021Offline Reinforcement Learning with Value-based Episodic Memorycited by 36
  7. 2020Learning Guidance Rewards with Trajectory-space Smoothingcited by 30
  8. 2013Experiments with Random Projectioncited by 27
  9. 2021Generalizable Episodic Memory for Deep Reinforcement Learningcited by 22
  10. 2021Automatic Web Testing using Curiosity-Driven Reinforcement Learningcited by 19
  11. 2022GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesiscited by 16
  12. 2018Fast deep reinforcement learning using online adjustments from the pastcited by 14
  13. 2021Model-Based Episodic Memory Induces Dynamic Hybrid Controlscited by 14
  14. 2019Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environmentscited by 13
  15. 2020KoGuN: Accelerating Deep Reinforcement Learning via Integrating Human Suboptimal Knowledgecited by 13
  16. 2021Solving Continuous Control with Episodic Memorycited by 7
  17. 2020Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Drivingcited by 5
  18. 2021Episodic Policy Gradient Trainingcited by 5
  19. 2020TOMA: Topological Map Abstraction for Reinforcement Learningcited by 1
  20. 2020Model-Free Episodic Control with State Aggregationcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 3.

7 papers.

  1. 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
  2. 2024Episodic Reinforcement Learning with Expanded State-reward Spacecited by 3
  3. 2025Trajectory-Class-Aware Multi-Agent Reinforcement Learningcited by 2
  4. 2024Subgoal Discovery Using a Free Energy Paradigm and State Aggregationscited by 1
  5. 2025Agentic Episodic Controlcited by 1
  6. 2023Robustness Verification of Deep Reinforcement Learning Based Control Systems using Reward Martingalescited by 0
  7. 2025Memory-Driven Self-Improvement for Decision Making with Large Language Modelscited by 0