When Does Adaptive Guidance Help? Belief-Aware Privileged Distillation for Autonomous Driving Under Partial Observability
Not on the map. Why?
Abstract (arXiv)
Guided Soft Actor-Critic (GSAC) distills knowledge from a privileged full-state teacher to a partial-observation student for autonomous driving, but uses a fixed distillation coefficient lambda regardless of the agent's uncertainty. We present Belief-Aware GSAC (BA-GSAC), which modulates lambda via ensemble disagreement, and use it as a testbed for a systematic empirical study asking: when does adaptive guidance actually help? Evaluating five strategies (fixed lambda in {0.01, 0.1}, adaptive, linear decay, and vanilla SAC) across three POMDP difficulty levels on Highway-Env, we find that preliminary single-seed runs suggest benefits under mild and moderate partial observability, but under severe occlusion (evaluated with 3 seeds for all methods) the adaptive coefficient collapses to lambda_min within about 3K steps. We trace this to an observability blindness phenomenon: because the ensemble predicts partial observations, it achieves low disagreement even under heavy occlusion, modeling what is visible but unable to detect what is missing. We diagnose the root cause and propose an architectural fix (training the ensemble on full-state predictions using the guiding actor's privileged access); while not validated here, we show that even with current limitations, the warmup phase provides measurable stabilization (CV=13.3% vs. 29.8% for constant lambda=0.01). In fact, a simple deterministic linear decay schedule achieves the best severe-POMDP performance across all metrics (mean 116.5, CV=8.9%), suggesting that the scheduling effect, not the ensemble, drives the stability benefit. These findings provide practical guidance for designing uncertainty-aware teacher-student frameworks and highlight ensemble prediction targets as an important design choice.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 24 May 2026.
Builds on
25 of 33 references.
References · 25
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
- 2018World Modelscited by 1,444
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
- 2018Exploration by Random Network Distillationcited by 966
- 2017Constrained Policy Optimizationcited by 928
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2016Deep Exploration via Bootstrapped DQNcited by 518
- 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
13 more
- 2020Planning to Explore via Self-Supervised World Modelscited by 365
- 2019Learning by Cheatingcited by 361
- 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
- 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
- 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
- 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 202
- 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
- 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 150
- 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
- 2019Distilling Policy Distillationcited by 81
- 2019Worst Cases Policy Gradientscited by 52
- 2021Unbiased Asymmetric Reinforcement Learning under Partial Observabilitycited by 24
- 2019Approximate Robust Control of Uncertain Dynamical Systemscited by 9
Built on by
Not cited yet.