mlsanityData to 7 Oct 2026

On Improving Deep Reinforcement Learning for POMDPs

arXiv 1804.06309 ↗First posted 17 Apr 2018cs.LG stat.MLcited by 49

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning (RL) recently emerged as one of the most competitive approaches for learning in sequential decision making problems with fully observable environments, e.g., computer Go. However, very little work has been done in deep RL to handle partially observable environments. We propose a new architecture called Action-specific Deep Recurrent Q-Network (ADRQN) to enhance learning performance in partially observable domains. Actions are encoded by a fully connected layer and coupled with a convolutional observation to form an action-observation pair. The time series of action-observation pairs are then integrated by an LSTM layer that learns latent states based on which a fully connected layer computes Q-values as in conventional Deep Q-Networks (DQNs). We demonstrate the effectiveness of our new architecture in several partially observable domains, including flickering Atari games.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 13; 2023: 9; 2024: 3; 2025: 7; 2026: 3.

49 papers.

Sort:

By area:

  1. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
  2. 2021Memory-based Deep Reinforcement Learning for POMDPscited by 53
  3. 2022Provable Reinforcement Learning with a Short-Term Memorycited by 40
  4. 2022Deep Transformer Q-Networks for Partially Observable Reinforcement Learningcited by 35
  5. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 34
  6. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 28
  7. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  8. 2021Structured World Belief for Reinforcement Learning in POMDPcited by 21
  9. 2021Recurrent Off-policy Baselines for Memory-based Continuous Controlcited by 19
  10. 2021Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domainscited by 18
  11. 2022A Bayesian Framework for Digital Twin-Based Control, Monitoring, and Data Collection in Wireless Systemscited by 18
  12. 2023POMDP inference and robust solution via deep reinforcement learning: An application to railway optimal maintenancecited by 13
37 more
  1. 2022Recurrent networks, hidden states and beliefs in partially observable environmentscited by 11
  2. 2022Bridging POMDPs and Bayesian decision making for robust maintenance planning under model uncertainty: An application to railway systemscited by 11
  3. 2023Informed POMDP: Leveraging Additional Information in Model-Based RLcited by 11
  4. 2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
  5. 2023Provable Representation with Efficient Planning for Partial Observable Reinforcement Learningcited by 10
  6. 2022Reinforcement Learning in Presence of Discrete Markovian Context Evolutioncited by 9
  7. 2022Discovering Exfiltration Paths Using Reinforcement Learning with Attack Graphscited by 8
  8. 2021SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learningcited by 7
  9. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutionscited by 7
  10. 2022Generalized Beliefs for Cooperative AIcited by 5
  11. 2025A Theoretical Justification for Asymmetric Actor-Critic Algorithmscited by 5
  12. 2026Cochain Perspectives on Temporal-Difference Signals for Learning Beyond Markov Dynamicscited by 5
  13. 2021How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control?cited by 4
  14. 2021Scientific Discovery and the Cost of Measurement -- Balancing Information and Cost in Reinforcement Learningcited by 4
  15. 2023On the Challenges of using Reinforcement Learning in Precision Drug Dosing: Delay and Prolongedness of Action Effectscited by 4
  16. 2025Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Accesscited by 2
  17. 2021Uncertainty Maximization in Partially Observable Domains: A Cognitive Perspectivecited by 1
  18. 2021Learning What to Memorize: Using Intrinsic Motivation to Form Useful Memory in Partially Observable Reinforcement Learningcited by 1
  19. 2021Towards Personalization of User Preferences in Partially Observable Smart Home Environmentscited by 1
  20. 2022Planning and Learning: Path-Planning for Autonomous Vehicles, a Review of the Literaturecited by 1
  21. 2022Efficient LSTM Training with Eligibility Tracescited by 1
  22. 2022Digital Twin-Based Multiple Access Optimization and Monitoring via Model-Driven Bayesian Learningcited by 1
  23. 2022Training Robots to Evaluate Robots: Example-Based Interactive Reward Functions for Policy Learningcited by 1
  24. 2023Model-free Motion Planning of Autonomous Agents for Complex Tasks in Partially Observable Environmentscited by 1
  25. 2023Robust Recurrent Reinforcement Learning under Evolving Hidden Disturbances with Application to Rover Wheel Slipcited by 1
  26. 2023An investigation of belief-free DRL and MCTS for inspection and maintenance planningcited by 1
  27. 2024Towards Cost Sensitive Decision Makingcited by 1
  28. 2025ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPscited by 1
  29. 2026Investigating Action Encodings in Recurrent Neural Networks in Reinforcement Learningcited by 1
  30. 2021Blockwise Sequential Model Learning for Partially Observable Reinforcement Learningcited by 0
  31. 2022Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarioscited by 0
  32. 2023End-to-End Policy Gradient Method for POMDPs and Explainable Agentscited by 0
  33. 2024Online Learning of Temporal Dependencies for Sustainable Foraging Problemcited by 0
  34. 2025Multi-source Plume Tracing via Multi-Agent Reinforcement Learningcited by 0
  35. 2025Bi-directional Recurrence Improves Transformer in Partially Observable Markov Decision Processescited by 0
  36. 2025Convergence of regularized agent-state-based Q-learning in POMDPscited by 0
  37. 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidancecited by 0