mlsanityData to 7 Oct 2026

Neural Map: Structured Memory for Deep Reinforcement Learning

arXiv 1702.08360 ↗First posted 27 Feb 2017cs.LGcited by 88

Not on the map. Why?

Abstract (arXiv)

A critical component to enabling intelligent reasoning in partially observable environments is memory. Despite this importance, Deep Reinforcement Learning (DRL) agents have so far used relatively simple memory architectures, with the main methods to overcome partial observability being either a temporal convolution over the past k frames or an LSTM layer. More recent work (Oh et al., 2016) has went beyond these architectures by using memory networks which can allow more sophisticated addressing schemes over the past k frames. But even these architectures are unsatisfactory due to the reason that they are limited to only remembering information from the last k frames. In this paper, we develop a memory system with an adaptable write operator that is customized to the sorts of 3D environments that DRL agents typically interact with. This architecture, called the Neural Map, uses a spatially structured 2D memory image to learn to store arbitrary information about the environment over long time lags. We demonstrate empirically that the Neural Map surpasses previous DRL memories on a set of challenging 2D and 3D maze environments and show that it is capable of generalizing to environments that were not seen during training.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 0; 2020: 0; 2021: 25; 2022: 12; 2023: 9; 2024: 13; 2025: 17; 2026: 10.

88 papers.

Sort:

By area:

  1. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
  2. 2022Navigating to Objects in the Real Worldcited by 104
  3. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  4. 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 64
  5. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  6. 2023Renderable Neural Radiance Map for Visual Navigationcited by 51
  7. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  8. 2021VTNet: Visual Transformer Network for Object Goal Navigationcited by 50
  9. 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 50
  10. 2021Scene-Intuitive Agent for Remote Embodied Visual Groundingcited by 49
  11. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48
  12. 2021Learning to Map for Active Semantic Goal Navigationcited by 47
38 more
  1. 2021Coupling Vision and Proprioception for Navigation of Legged Robotscited by 46
  2. 2022Retrospectives on the Embodied AI Workshopcited by 46
  3. 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
  4. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  5. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 34
  6. 2024Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Modelscited by 32
  7. 2025CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debuggingcited by 31
  8. 2018Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learningcited by 27
  9. 2022Towards Versatile Embodied Navigationcited by 26
  10. 2021Differentiable Spatial Planning using Transformerscited by 25
  11. 2022Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learningcited by 22
  12. 2022Multi-Object Navigation with dynamically learned neural implicit representationscited by 21
  13. 2021Learning Efficient Multi-Agent Cooperative Visual Explorationcited by 19
  14. 2021Collaborative Visual Navigationcited by 18
  15. 2021A Survey on Deep Reinforcement Learning for Audio-Based Applicationscited by 17
  16. 2021Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigationcited by 17
  17. 2023End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenoncited by 16
  18. 2022Navigating to Objects in Unseen Environments by Distance Predictioncited by 15
  19. 2025R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memorycited by 14
  20. 2024MemoNav: Working Memory Model for Visual Navigationcited by 13
  21. 2021Environment Predictive Coding for Embodied Agentscited by 12
  22. 2021MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigationcited by 11
  23. 2022Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomescited by 11
  24. 2021Agent-Centric Relation Graph for Object Visual Navigationcited by 10
  25. 2025Curriculum Learning-Driven PIELMs for Fluid Flow Simulationscited by 8
  26. 2023Sample-Efficient Learning to Solve a Real-World Labyrinth Game Using Data-Augmented Model-Based Reinforcement Learningcited by 7
  27. 2025LoopNav: Benchmarking Spatial Consistency in World Modelscited by 7
  28. 2021Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decompositioncited by 6
  29. 2023Learning with a Mole: Transferable latent spatial representations for navigation without reconstructioncited by 6
  30. 2025Inverse Reinforcement Learning with Switching Rewards and History Dependency for Characterizing Animal Behaviorscited by 6
  31. 2025ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problemscited by 6
  32. 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environmentscited by 6
  33. 2021Graph augmented Deep Reinforcement Learning in the GameRLand3D environmentcited by 5
  34. 2023Multi-goal Audio-visual Navigation using Sound Direction Mapcited by 5
  35. 2024Learning to navigate efficiently and precisely in real environmentscited by 5
  36. 2024Unraveling the Complexity of Memory in RL Agents: an Approach for Classification and Evaluationcited by 5
  37. 2025STMA: A Spatio-Temporal Memory Agent for Long-Horizon Embodied Task Planningcited by 5
  38. 2021Towards real-world navigation with deep differentiable plannerscited by 4

50 most cited of 88 All 88 (JSON)