# Robust Asymmetric Learning in POMDPs

arXiv: https://arxiv.org/abs/2012.15566 · HTML: https://mlsanity.com/paper/2012.15566/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2020.

## Built on by (most cited first, 23 of 23)

- [Learning Visible Connectivity Dynamics for Cloth Smoothing](https://mlsanity.com/paper/2105.10389/) (2021)
- [Hindsight Learning for MDPs with Exogenous Inputs](https://mlsanity.com/paper/2207.06272/) (2022)
- [LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving](https://mlsanity.com/paper/2512.20563/) (2025)
- [GridToPix: Training Embodied Agents with Minimal Supervision](https://mlsanity.com/paper/2105.00931/) (2021)
- [Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs](https://mlsanity.com/paper/2402.08733/) (2024)
- [Leveraging Fully Observable Policies for Learning under Partial Observability](https://mlsanity.com/paper/2211.01991/) (2022)
- [Learning in POMDPs is Sample-Efficient with Hindsight Observability](https://mlsanity.com/paper/2301.13857/) (2023)
- [TGRL: An Algorithm for Teacher Guided Reinforcement Learning](https://mlsanity.com/paper/2307.03186/) (2023)
- [Informed POMDP: Leveraging Additional Information in Model-Based RL](https://mlsanity.com/paper/2306.11488/) (2023)
- [Bridging the Sim-to-Real Gap from the Information Bottleneck Perspective](https://mlsanity.com/paper/2305.18464/) (2023)
- [Improved Policy Optimization for Online Imitation Learning](https://mlsanity.com/paper/2208.00088/) (2022)
- [Student-Informed Teacher Training](https://mlsanity.com/paper/2412.09149/) (2024)
- [A Theoretical Justification for Asymmetric Actor-Critic Algorithms](https://mlsanity.com/paper/2501.19116/) (2025)
- [AgentMixer: Multi-Agent Correlated Policy Factorization](https://mlsanity.com/paper/2401.08728/) (2024)
- [Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learning](https://mlsanity.com/paper/2106.09256/) (2021)
- [Guided Policy Optimization under Partial Observability](https://mlsanity.com/paper/2505.15418/) (2025)
- [Multi-Agent Guided Policy Optimization](https://mlsanity.com/paper/2507.18059/) (2025)
- [To Distill or Decide? Understanding the Algorithmic Trade-off in Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2510.03207/) (2025)
- [Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneity](https://mlsanity.com/paper/2404.07266/) (2024)
- [Distilling Realizable Students from Unrealizable Teachers](https://mlsanity.com/paper/2505.09546/) (2025)
- [Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access](https://mlsanity.com/paper/2509.26000/) (2025)
- [Communication-Enhanced Tutoring for Efficient Decentralized Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.13661/) (2025)
- [Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance](https://mlsanity.com/paper/2607.26040/) (2026)

## Abstract (arXiv)

Policies for partially observed Markov decision processes can be efficiently learned by imitating policies for the corresponding fully observed Markov decision processes. Unfortunately, existing approaches for this kind of imitation learning have a serious flaw: the expert does not know what the trainee cannot see, and so may encourage actions that are sub-optimal, even unsafe, under partial information. We derive an objective to instead train the expert to maximize the expected reward of the imitating agent policy, and use it to construct an efficient algorithm, adaptive asymmetric DAgger (A2D), that jointly trains the expert and the agent. We show that A2D produces an expert policy that the agent can safely imitate, in turn outperforming policies learned by imitating a fixed expert.