mlsanityData to 14 Oct 2026

Guided Policy Optimization under Partial Observability

arXiv 2505.15418 ↗First posted 21 May 2025cs.LG cs.AI cs.ROcited by 3

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Reinforcement Learning (RL) in partially observable environments poses significant challenges due to the complexity of learning under uncertainty. While additional information, such as that available in simulations, can enhance training, effectively leveraging it remains an open problem. To address this, we introduce Guided Policy Optimization (GPO), a framework that co-trains a guider and a learner. The guider takes advantage of privileged information while ensuring alignment with the learner's policy that is primarily trained via imitation learning. We theoretically demonstrate that this learning scheme achieves optimality comparable to direct RL, thereby overcoming key limitations inherent in existing approaches. Empirical evaluations show strong performance of GPO across various tasks, including continuous control with partial observability and noise, and memory-based challenges, significantly outperforming existing methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 42 of its 57 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

42 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Provable Partially Observable Reinforcement Learning with Privileged Information1 Dec

Other references · 41

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  7. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  8. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  9. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  10. 2018Behavioral Cloning from Observationcited by 492
  11. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  12. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
29 more
  1. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  2. 2019Causal Confusion in Imitation Learningcited by 288
  3. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  4. 2021A System for General In-Hand Object Re-Orientationcited by 165
  5. 2018Deep Variational Reinforcement Learning for POMDPscited by 125
  6. 2022On the Convergence Rates of Policy Gradient Methodscited by 119
  7. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 118
  8. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
  9. 2020Mirror Descent Policy Optimizationcited by 106
  10. 2022When Is Partially Observable Reinforcement Learning Not Scary?cited by 100
  11. 2019Distilling Policy Distillationcited by 84
  12. 2023Multi-View Masked World Models for Visual Robotic Manipulationcited by 80
  13. 2023Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorldcited by 65
  14. 2023POPGym: Benchmarking Partially Observable Reinforcement Learningcited by 62
  15. 2021Memory-based Deep Reinforcement Learning for POMDPscited by 54
  16. 2015Learning to Search Better Than Your Teachercited by 50
  17. 2022Provable Reinforcement Learning with a Short-Term Memorycited by 40
  18. 2020Bridging the Imitation Gap by Adaptive Insubordinationcited by 38
  19. 2017Learning Heuristic Search via Imitationcited by 34
  20. 2018Deep Learning under Privileged Information Using Heteroscedastic Dropoutcited by 27
  21. 2020Robust Imitation Learning from Noisy Demonstrationscited by 26
  22. 2021Unbiased Asymmetric Reinforcement Learning under Partial Observabilitycited by 25
  23. 2020Robust Asymmetric Learning in POMDPscited by 23
  24. 2022Leveraging Fully Observable Policies for Learning under Partial Observabilitycited by 18
  25. 2023Learning in POMDPs is Sample-Efficient with Hindsight Observabilitycited by 18
  26. 2023TGRL: An Algorithm for Teacher Guided Reinforcement Learningcited by 17
  27. 2016Guided Policy Search as Approximate Mirror Descentcited by 15
  28. 2017Adaptive Information Gathering via Imitation Learningcited by 6
  29. 2019Co-training for Policy Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Multi-Agent Guided Policy Optimizationcited by 3
  2. 2025To Distill or Decide? Understanding the Algorithmic Trade-off in Partially Observable Reinforcement Learningcited by 3
  3. 2026Belief-State RWKV for Reinforcement Learning under Partial Observabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domains
  2. 2026Learning from the Near Future: Temporal Self-Distillation for RLVR
  3. 2025Reinforcement Learning via Implicit Imitation Guidance
  4. 2026PriPG-RL: Privileged Planner-Guided Reinforcement Learning for Partially Observable Systems with Anytime-Feasible MPC
  5. 2025Learning safe, constrained policies via imitation learning: Connection to Probabilistic Inference and a Naive Algorithm
  6. 2024Diffusion Policy Policy Optimization
  7. 2025Policy Gradient Guidance Enables Test Time Control
  8. 2018Fast Policy Learning through Imitation and Reinforcement