mlsanityData to 14 Oct 2026

Zero-Shot Reinforcement Learning Under Partial Observability

arXiv 2506.15446 ↗First posted 18 Jun 2025cs.LG cs.AIcited by 6

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Recent work has shown that, under certain assumptions, zero-shot reinforcement learning (RL) methods can generalise to any unseen task in an environment after reward-free pre-training. Access to Markov states is one such assumption, yet, in many real-world applications, the Markov state is only partially observable. Here, we explore how the performance of standard zero-shot RL methods degrades when subjected to partially observability, and show that, as in single-task RL, memory-based architectures are an effective remedy. We evaluate our memory-based zero-shot RL methods in domains where the states, rewards and a change in dynamics are partially observed, and show improved performance over memory-free baselines. Our code is open-sourced via: https://enjeeneer.io/projects/bfms-with-memory/.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 61 of its 91 references that are in the map. First posted on arXiv 18 Jun 2025.

Builds on

61 of 91 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers17 Nov
  2. 2024Foundation Policies with Hilbert Representations23 Feb

Other references · 59

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2016Layer Normalizationcited by 5,669
  6. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  9. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  10. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  11. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  12. 2018World Modelscited by 1,513
38 more
  1. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  3. 2023Mastering Diverse Domains through World Modelscited by 1,253
  4. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  5. 2020Mastering Atari with Discrete World Modelscited by 1,007
  6. 2018DeepMind Control Suitecited by 956
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  8. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  9. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  10. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  11. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  12. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 424
  13. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  14. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  15. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  16. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  17. 2015Contextual Markov Decision Processescited by 220
  18. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  19. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  20. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
  21. 2018Assessing Generalization in Deep Reinforcement Learningcited by 133
  22. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 125
  23. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  24. 2018Generalization and Regularization in DQNcited by 118
  25. 2021Learning One Representation to Optimize All Rewardscited by 115
  26. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
  27. 2018Universal Successor Features Approximatorscited by 103
  28. 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 98
  29. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 96
  30. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 83
  31. 2015Memory-based control with recurrent neural networkscited by 78
  32. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 78
  33. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  34. 2021Learning Successor States and Goal-Dependent Values: A Mathematical Viewpointcited by 70
  35. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regressioncited by 66
  36. 2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 62
  37. 2019Hyperbolic Discounting and Learning over Multiple Horizonscited by 60
  38. 2021Memory-based Deep Reinforcement Learning for POMDPscited by 54

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026Discovering What You Can Control: Interventional Boundary Discovery for Reinforcement Learningcited by 2
  2. 2026A Reward-Free Viewpoint on Multi-Objective Reinforcement Learningcited by 2
  3. 2026From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learningcited by 1
  4. 2025On Swarm Leader Identification using Probing Policiescited by 0
  5. 2026Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agentscited by 0
  6. 2026When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisitedcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Unified Framework for Zero-Shot Reinforcement Learning
  2. 2025Tackling the Zero-Shot Reinforcement Learning Loss Directly
  3. 2026Improving Zero-Shot Offline RL via Behavioral Task Sampling
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
  5. 2025Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domains
  6. 2026Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilities
  7. 2024Zero-Shot Reinforcement Learning via Function Encoders
  8. 2024Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learning