mlsanityData to 14 Oct 2026

Generalization of Reinforcement Learners with Working and Episodic Memory

arXiv 1910.13406 ↗First posted 29 Oct 2019cs.LG cs.AI stat.MLcited by 36

Not on the map. Why?

Abstract (arXiv)

Memory is an important aspect of intelligence and plays a role in many deep reinforcement learning models. However, little progress has been made in understanding when specific memory systems help more than others and how well they generalize. The field also has yet to see a prevalent consistent and rigorous approach for evaluating agent performance on holdout data. In this paper, we aim to develop a comprehensive methodology to test different kinds of memory in an agent and assess how well the agent can apply what it learns in training to a holdout set that differs from the training set along dimensions that we suggest are relevant for evaluating memory-specific generalization. To that end, we first construct a diverse set of memory tasks that allow us to evaluate test-time generalization across multiple dimensions. Second, we develop and perform multiple ablations on an agent architecture that combines multiple memory systems, observe its baseline models, and investigate its performance against the task suite.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 10; 2023: 6; 2024: 5; 2025: 4; 2026: 2.

36 papers.

Sort:

By area:

  1. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  2. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  3. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
  4. 2023When Do Transformers Shine in RL? Decoupling Memory from Credit Assignmentcited by 63
  5. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  6. 2022Evaluating Long-Term Memory in 3D Mazescited by 43
  7. 2022Transformers are Meta-Reinforcement Learnerscited by 41
  8. 2021Towards mental time travel: a hierarchical memory for reinforcement learning agentscited by 40
  9. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 36
  10. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  11. 2021Synthetic Returns for Long-Term Credit Assignmentcited by 27
  12. 2022On Neural Architecture Inductive Biases for Relational Taskscited by 26
24 more
  1. 2021CoBERL: Contrastive BERT for Reinforcement Learningcited by 21
  2. 2021Active Hierarchical Exploration with Stable Subgoal Representation Learningcited by 17
  3. 2023Semantic HELM: A Human-Readable Memory for Reinforcement Learningcited by 17
  4. 2023Recurrent Hypernetworks are Surprisingly Strong in Meta-RLcited by 16
  5. 2023Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agentscited by 16
  6. 2021Model-Based Episodic Memory Induces Dynamic Hybrid Controlscited by 14
  7. 2022Generalization, Mayhems and Limits in Recurrent Proximal Policy Optimizationcited by 10
  8. 2021Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learningcited by 6
  9. 2022DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Bodycited by 6
  10. 2024Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observabilitycited by 5
  11. 2024Unraveling the Complexity of Memory in RL Agents: an Approach for Classification and Evaluationcited by 5
  12. 2021Differentiable Architecture Search for Reinforcement Learningcited by 4
  13. 2025Investigating Memory in Model-Free RL with POPGym Arcadecited by 3
  14. 2022Memory-Augmented Graph Neural Networks: A Brain-Inspired Reviewcited by 2
  15. 2023Grid Cell-Inspired Fragmentation and Recall for Efficient Map Buildingcited by 2
  16. 2024SplAgger: Split Aggregation for Meta-Reinforcement Learningcited by 2
  17. 2024Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Mapscited by 2
  18. 2022Modeling Human Behavior Part I -- Learning and Belief Approachescited by 1
  19. 2024Treating Brain-inspired Memories as Priors for Diffusion Model to Forecast Multivariate Time Seriescited by 1
  20. 2025Agentic Episodic Controlcited by 1
  21. 2022Extended Intelligencecited by 0
  22. 2025Momentum Boosted Episodic Memory for Improving Learning in Long-Tailed RL Environmentscited by 0
  23. 2026ALER: Adaptive Learnable Experience Rewriting for Reinforcement Learningcited by 0
  24. 2026Does Learning Recurrent Networks Matter in Partially Observable RL?cited by 0