mlsanityData to 14 Oct 2026

Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access

arXiv 2509.26000 ↗First posted 30 Sep 2025cs.LG stat.MLcited by 2

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Asymmetric reinforcement learning leverages privileged information available during training to improve learning under partial observability. Existing asymmetric actor-critic methods typically assume access to the full environment state to condition the critic during training, which is often unrealistic in practice. We introduce the informed asymmetric actor-critic framework that allows the critic to be conditioned on arbitrary state-dependent privileged signals, and show that any such signal yields unbiased policy gradient estimates. This substantially expands the set of admissible privileged information and raises the problem of selecting the most informative signals for learning. To this end, we propose two novel informativeness criteria: a dependence-based test that can be applied prior to training, and a test based on improvements in value prediction that can be applied post hoc. Experiments on partially observable benchmarks and synthetic environments demonstrate that carefully selected privileged signals can match or outperform full-state asymmetric baselines while relying on strictly less state information.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 49 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

28 of 49 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025A Theoretical Justification for Asymmetric Actor-Critic Algorithms31 Jan
  2. 2024Provable Partially Observable Reinforcement Learning with Privileged Information1 Dec
  3. 2024Recurrent Natural Policy Gradient for POMDPs28 May

Other references · 25

  1. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  2. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  3. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  4. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 225
  5. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  6. 2019Neural Policy Gradient Methods: Global Optimality and Rates of Convergencecited by 159
  7. 2018Deep Variational Reinforcement Learning for POMDPscited by 125
  8. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 102
  9. 2020A Measure-Theoretic Approach to Kernel Conditional Mean Embeddingscited by 86
  10. 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 70
  11. 2019Shaping Belief States with Generative Environment Models for RLcited by 66
  12. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 54
13 more
  1. 2018On Improving Deep Reinforcement Learning for POMDPscited by 50
  2. 2018Learning and Querying Fast Generative Models for Reinforcement Learningcited by 48
  3. 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 47
  4. 2018Neural Predictive Belief Representationscited by 40
  5. 2017Data-driven Planning via Imitation Learningcited by 36
  6. 2021Unbiased Asymmetric Reinforcement Learning under Partial Observabilitycited by 25
  7. 2020Robust Asymmetric Learning in POMDPscited by 23
  8. 2024Privileged Sensing Scaffolds Reinforcement Learningcited by 19
  9. 2017On overfitting and asymptotic bias in batch reinforcement learning with partial observabilitycited by 16
  10. 2024A Super-human Vision-based Reinforcement Learning Agent for Autonomous Racing in Gran Turismocited by 15
  11. 2023Informed POMDP: Leveraging Additional Information in Model-Based RLcited by 11
  12. 2023The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Modelscited by 9
  13. 2023Nystr"om M-Hilbert-Schmidt Independence Criterioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidancecited by 0
  2. 2026Shared Actors Need Not Share Critics: Effects of Value Mismatch in Parallel Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Memory-State Critic for Asymmetric Actor-Critic with Application to Vision-Based Pursuit-Evasion
  2. 2022Beyond the Policy Gradient Theorem for Efficient Policy Updates in Actor-Critic Algorithms
  3. 2025Bidirectional Soft Actor-Critic: Leveraging Forward and Reverse KL Divergence for Efficient Reinforcement Learning
  4. 2025Actor-Critic without Actor
  5. 2026Hidden States as Value Gradients: The Pontryagin Structure of Recurrent Policies
  6. 2025Overcoming Non-stationary Dynamics with Evidential Proximal Policy Optimization
  7. 2025PIGDreamer: Privileged Information Guided World Models for Safe Partially Observable Reinforcement Learning
  8. 2025Distributions as Actions: A Unified Framework for Diverse Action Spaces