mlsanityData to 14 Oct 2026

Memory-based control with recurrent neural networks

arXiv 1512.04455 ↗First posted 14 Dec 2015cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

Partially observed control problems are a challenging aspect of reinforcement learning. We extend two related, model-free algorithms for continuous control -- deterministic policy gradient and stochastic value gradient -- to solve partially observed domains using recurrent neural networks trained with backpropagation through time. We demonstrate that this approach, coupled with long-short term memory is able to solve a variety of physical control problems exhibiting an assortment of memory requirements. These include the short-term integration of information from noisy sensors and the identification of system parameters, as well as long-term memory problems that require preserving information over many time steps. We also demonstrate success on a combined exploration and memory problem in the form of a simplified version of the well-known Morris water maze task. Finally, we show that our approach can deal with high-dimensional observations by learning directly from pixels. We find that recurrent deterministic and stochastic policies are able to learn similarly good solutions to these tasks, including the water maze where the agent must learn effective search strategies.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Dec 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 10; 2023: 21; 2024: 7; 2025: 6; 2026: 11.

78 papers.

Sort:

By area:

  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  2. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  3. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 81
  4. 2021Memory-based Deep Reinforcement Learning for POMDPscited by 54
  5. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 41
  6. 2021Unsupervised Visual Attention and Invariance for Reinforcement Learningcited by 30
  7. 2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
  8. 2021Recurrent Off-policy Baselines for Memory-based Continuous Controlcited by 21
  9. 2024Equivariant Reinforcement Learning under Partial Observabilitycited by 18
  10. 2021Recurrent Neural Network Controllers Synthesis with Stability Guarantees for Partially Observed Systemscited by 16
  11. 2023Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agentscited by 16
  12. 2023AdaptNet: Policy Adaptation for Physics-Based Character Controlcited by 16
38 more
  1. 2021Analysis of a Target-Based Actor-Critic Algorithm with Linear Function Approximationcited by 14
  2. 2022Recurrent networks, hidden states and beliefs in partially observable environmentscited by 12
  3. 2022Task-Agnostic Continual Reinforcement Learning: Gaining Insights and Overcoming Challengescited by 11
  4. 2023Informed POMDP: Leveraging Additional Information in Model-Based RLcited by 11
  5. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Surveycited by 11
  6. 2022Generalization, Mayhems and Limits in Recurrent Proximal Policy Optimizationcited by 10
  7. 2023Provable Representation with Efficient Planning for Partial Observable Reinforcement Learningcited by 10
  8. 2023ODE-based Recurrent Model-free Reinforcement Learning for POMDPscited by 7
  9. 2023Offline RL with Observation Histories: Analyzing and Improving Sample Complexitycited by 7
  10. 2024Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancycited by 7
  11. 2022Autonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Reviewcited by 6
  12. 2025A Theoretical Justification for Asymmetric Actor-Critic Algorithmscited by 6
  13. 2025Zero-Shot Reinforcement Learning Under Partial Observabilitycited by 6
  14. 2024Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observabilitycited by 5
  15. 2021Self-optimizing adaptive optics control with Reinforcement Learning for high-contrast imagingcited by 4
  16. 2022RL-PGO: Reinforcement Learning-based Planar Pose-Graph Optimizationcited by 4
  17. 2023The configurable tree graph (CT-graph): measurable problems in partially observable and distal reward environments for lifelong reinforcement learningcited by 4
  18. 2023PID-Inspired Inductive Biases for Deep Reinforcement Learning in Partially Observable Control Taskscited by 4
  19. 2023Decoupling Meta-Reinforcement Learning with Gaussian Task Contexts and Skillscited by 4
  20. 2021A Deep Reinforcement Learning Framework for Eco-driving in Connected and Automated Hybrid Electric Vehiclescited by 3
  21. 2021Deep Reinforcement Learning for Safe Landing Site Selection with Concurrent Consideration of Divert Maneuverscited by 3
  22. 2021Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learningcited by 3
  23. 2021Reinforcement Learning using Guided Observabilitycited by 3
  24. 2022Effective Multi-User Delay-Constrained Scheduling with Deep Recurrent Reinforcement Learningcited by 3
  25. 2022Spatial-temporal recurrent reinforcement learning for autonomous shipscited by 3
  26. 2023Finite-Time Analysis of Minimax Q-Learning for Two-Player Zero-Sum Markov Games: Switching System Approachcited by 3
  27. 2021Robust Reinforcement Learning under model misspecificationcited by 2
  28. 2021Graph Convolutional Memory using Topological Priorscited by 2
  29. 2022Distributional Reinforcement Learning for Scheduling of Chemical Production Processescited by 2
  30. 2023Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learningcited by 2
  31. 2023Learning Computational Efficient Bots with Costly Featurescited by 2
  32. 2023Time-Efficient Reinforcement Learning with Stochastic Stateful Policiescited by 2
  33. 2024Navigation with QPHIL: Quantizing Planner for Hierarchical Implicit Q-Learningcited by 2
  34. 2021Cross-Modal Contrastive Learning of Representations for Navigation using Lightweight, Low-Cost Millimeter Wave Radar for Adverse Environmental Conditionscited by 1
  35. 2021Hierarchical RNNs-Based Transformers MADDPG for Mixed Cooperative-Competitive Environmentscited by 1
  36. 2021Inducing Functions through Reinforcement Learning without Task Specificationcited by 1
  37. 2022Recursive Least Squares Advantage Actor-Critic Algorithmscited by 1
  38. 2023Model-free Motion Planning of Autonomous Agents for Complex Tasks in Partially Observable Environmentscited by 1

50 most cited of 78 All 78 (JSON)