mlsanityData to 7 Oct 2026

Uncertainty-Aware Decision Transformer for Stochastic Driving Environments

arXiv 2309.16397 ↗First posted 28 Sep 2023cs.LG cs.AIcited by 8

On the map: End-to-end autonomous driving → Reinforcement learning for driving

also in 1 more
Abstract (arXiv)

Offline Reinforcement Learning (RL) enables policy learning without active interactions, making it especially appealing for self-driving tasks. Recent successes of Transformers inspire casting offline RL as sequence modeling, which, however, fails in stochastic environments with incorrect assumptions that identical actions can consistently achieve the same goal. In this paper, we introduce an UNcertainty-awaRE deciSion Transformer (UNREST) for planning in stochastic driving environments without introducing additional transition or complex generative models. Specifically, UNREST estimates uncertainties by conditional mutual information between transitions and returns. Discovering 'uncertainty accumulation' and 'temporal locality' properties of driving environments, we replace the global returns in decision transformers with truncated returns less affected by environments to learn from actual outcomes of actions rather than environment transitions. We also dynamically evaluate uncertainty at inference for cautious planning. Extensive experiments demonstrate UNREST's superior performance in various driving scenarios and the power of our uncertainty estimation strategy.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 52 of its 78 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

52 of 78 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization17 Feb
  2. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement13 Oct

Other references · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  9. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  10. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  11. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  12. 2019Adversarial Training for Free!cited by 638
38 more
  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  2. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  3. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
  4. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  5. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  6. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  7. 2018Understanding disentangling in β-VAEcited by 332
  8. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  9. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  10. 2017Advances in Variational Inferencecited by 236
  11. 2022Multi-Game Decision Transformerscited by 212
  12. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  13. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  14. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  15. 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 125
  16. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  17. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  18. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
  19. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  20. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 73
  21. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 70
  22. 2023Elastic Decision Transformercited by 66
  23. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 62
  24. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learningcited by 48
  25. 2022Dichotomy of Control: Separating What You Can Control from What You Cannotcited by 47
  26. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
  27. 2016On the Quantitative Analysis of Decoder-Based Generative Modelscited by 40
  28. 2020Planning from Pixels using Inverse Dynamics Modelscited by 36
  29. 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
  30. 2022InterSim: Interactive Traffic Simulation via Explicit Relation Modelingcited by 27
  31. 2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 26
  32. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioningcited by 24
  33. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planningcited by 23
  34. 2022Addressing Optimism Bias in Sequence Modeling for Reinforcement Learningcited by 22
  35. 2022Imitating Past Successes can be Very Suboptimalcited by 20
  36. 2022Latent Discriminant deterministic Uncertaintycited by 17
  37. 2022Upside-Down Reinforcement Learning Can Diverge in Stochastic Environments With Episodic Resetscited by 10
  38. 2023Decision Transformer under Random Frame Droppingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 1.

8 papers.

  1. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  2. 2024SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Modelscited by 28
  3. 2025Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstractioncited by 4
  4. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  5. 2025Mirroring Users: Towards Building Preference-aligned User Simulator with User Feedback in Recommendationcited by 2
  6. 2025Energy-Efficient Deep Reinforcement Learning with Spiking Transformerscited by 0
  7. 2025An Uncertainty-Weighted Decision Transformer for Navigation in Dense, Complex Driving Scenarioscited by 0
  8. 2026Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving
  2. 2024Diffusion Policies for Risk-Averse Behavior Modeling in Offline Reinforcement Learning
  3. 2023Environment Transformer and Policy Optimization for Model-Based Offline Reinforcement Learning
  4. 2023Selective Uncertainty Propagation in Offline RL
  5. 2026A Risk-Sensitive and Uncertainty-Aware Decision-Making and Control Framework for Safe and Robust Autonomous Driving
  6. 2024RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes
  7. 2021Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning
  8. 2025From Imitation to Optimization: A Comparative Study of Offline Learning for Autonomous Driving