# Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access

arXiv: https://arxiv.org/abs/2509.26000 · HTML: https://mlsanity.com/paper/2509.26000/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 28 of 28)

- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model](https://mlsanity.com/paper/1907.00953/) (2019)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Neural Policy Gradient Methods: Global Optimality and Rates of Convergence](https://mlsanity.com/paper/1909.01150/) (2019)
- [Deep Variational Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1806.02426/) (2018)
- [Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning](https://mlsanity.com/paper/2004.14646/) (2020)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Approximate information state for approximate planning and reinforcement learning in partially observed systems](https://mlsanity.com/paper/2010.08843/) (2020)
- [Shaping Belief States with Generative Environment Models for RL](https://mlsanity.com/paper/1906.09237/) (2019)
- [Bridging State and History Representations: Understanding Self-Predictive RL](https://mlsanity.com/paper/2401.08898/) (2024)
- [On Improving Deep Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1804.06309/) (2018)
- [Learning and Querying Fast Generative Models for Reinforcement Learning](https://mlsanity.com/paper/1802.03006/) (2018)
- [Variational Recurrent Models for Solving Partially Observable Control Tasks](https://mlsanity.com/paper/1912.10703/) (2019)
- [Neural Predictive Belief Representations](https://mlsanity.com/paper/1811.06407/) (2018)
- [Data-driven Planning via Imitation Learning](https://mlsanity.com/paper/1711.06391/) (2017)
- [Unbiased Asymmetric Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2105.11674/) (2021)
- [Robust Asymmetric Learning in POMDPs](https://mlsanity.com/paper/2012.15566/) (2020)
- [Privileged Sensing Scaffolds Reinforcement Learning](https://mlsanity.com/paper/2405.14853/) (2024)
- [On overfitting and asymptotic bias in batch reinforcement learning with partial observability](https://mlsanity.com/paper/1709.07796/) (2017)
- [A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismo](https://mlsanity.com/paper/2406.12563/) (2024)
- [Provable Partially Observable Reinforcement Learning with Privileged Information](https://mlsanity.com/paper/2412.00985/) (2024)
- [Informed POMDP: Leveraging Additional Information in Model-Based RL](https://mlsanity.com/paper/2306.11488/) (2023)
- [The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Models](https://mlsanity.com/paper/2303.03284/) (2023)
- [Recurrent Natural Policy Gradient for POMDPs](https://mlsanity.com/paper/2405.18221/) (2024)
- [A Theoretical Justification for Asymmetric Actor-Critic Algorithms](https://mlsanity.com/paper/2501.19116/) (2025)
- [Nystr"om M-Hilbert-Schmidt Independence Criterion](https://mlsanity.com/paper/2302.09930/) (2023)

## Built on by (most cited first, 2 of 2)

- [Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance](https://mlsanity.com/paper/2607.26040/) (2026)
- [Shared Actors Need Not Share Critics: Effects of Value Mismatch in Parallel Reinforcement Learning](https://mlsanity.com/paper/2608.26481/) (2026)

## Abstract (arXiv)

Asymmetric reinforcement learning leverages privileged information available during training to improve learning under partial observability. Existing asymmetric actor-critic methods typically assume access to the full environment state to condition the critic during training, which is often unrealistic in practice. We introduce the informed asymmetric actor-critic framework that allows the critic to be conditioned on arbitrary state-dependent privileged signals, and show that any such signal yields unbiased policy gradient estimates. This substantially expands the set of admissible privileged information and raises the problem of selecting the most informative signals for learning. To this end, we propose two novel informativeness criteria: a dependence-based test that can be applied prior to training, and a test based on improvements in value prediction that can be applied post hoc. Experiments on partially observable benchmarks and synthetic environments demonstrate that carefully selected privileged signals can match or outperform full-state asymmetric baselines while relying on strictly less state information.