mlsanityData to 14 Oct 2026

Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving

arXiv 2605.30576 ↗First posted 28 May 2026cs.AIcited by 0

On the map: End-to-end autonomous driving → Reinforcement learning for driving

also in 1 more
Abstract (arXiv)

Exploration in reinforcement learning for autonomous driving is inherently unsafe: agents must experience novel behaviors to learn, yet exploration can lead to collisions or off-road driving. We propose an uncertainty-aware framework that leverages expert advice to guide exploration while avoiding long-term dependence. Advice is triggered when epistemic or aleatoric uncertainty exceeds adaptive thresholds derived from rolling buffers, ensuring advice evolves with the agent's confidence. A commitment-cooldown strategy with a stochastic early-stop heuristic regulates the duration and frequency of guidance, exposing the agent to coherent maneuvers without exhausting the advice budget. Expert and agent experiences are combined in a shared replay buffer within an off-policy implicit quantile network (IQN) backbone, enabling efficient reuse of expert trajectories. Experiments in CARLA show that our method outperforms the IQN baseline, improving success by 5-7% and reducing failures, demonstrating that risk-sensitive uncertainty coupled with regulated expert integration enables safer and more efficient exploration for sensor-based RL policy learning in unsignalized intersection navigation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 29 of its 41 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

29 of 41 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions20 Sep
  2. 2024HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving6 Jan
  3. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Driving16 Nov
  4. 2021Safe Driving via Expert Guided Policy Optimization13 Oct
  5. 2021Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning15 Jul
  6. 2021Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving21 May
  7. 2021Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning5 Feb

Other references · 22

  1. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  4. 2016Deep Exploration via Bootstrapped DQNcited by 527
  5. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  6. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  7. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 266
  8. 2022A Review of Uncertainty for Deep Reinforcement Learningcited by 40
  9. 2018Reinforced Imitation: Sample Efficient Deep Reinforcement Learning for Map-less Navigation by Leveraging Prior Demonstrationscited by 38
  10. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 35
  11. 2019AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teacherscited by 28
  12. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 28
10 more
  1. 2025Learning from Active Human Involvement through Proxy Value Propagationcited by 19
  2. 2023Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functionscited by 13
  3. 2023Guarded Policy Optimization with Imperfect Online Demonstrationscited by 10
  4. 2020Student-Initiated Action Advising via Advice Noveltycited by 7
  5. 2023Value Functions are Control Barrier Functions: Verification of Safe Policies using Control Theorycited by 6
  6. 2024Safe Reinforcement Learning in Black-Box Environments via Adaptive Shieldingcited by 6
  7. 2025IN-RIL: Interleaved Reinforcement and Imitation Learning for Policy Fine-Tuningcited by 5
  8. 2023Reducing Safety Interventions in Provably Safe Reinforcement Learningcited by 2
  9. 2023Learning to Recover for Safe Reinforcement Learningcited by 1
  10. 2023Agent-Aware Training for Agent-Agnostic Action Advising in Deep Reinforcement Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Uncertainty-Aware Decision Transformer for Stochastic Driving Environments
  2. 2026Uncertainty-Driven Self-Play for Robust Multi-Agent Driving Policies
  3. 2024RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes
  4. 2026Self-Aware Active Learning Enables Continual Improvement in Autonomous Driving
  5. 2026A Risk-Sensitive and Uncertainty-Aware Decision-Making and Control Framework for Safe and Robust Autonomous Driving
  6. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
  7. 2025InDRiVE: Intrinsic Disagreement based Reinforcement for Vehicle Exploration through Curiosity Driven Generalized World Model
  8. 2025InDRiVE: Reward-Free World-Model Pretraining for Autonomous Driving via Latent Disagreement