mlsanityData to 7 Oct 2026

Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving

arXiv 2311.10747 ↗First posted 31 Oct 2023cs.RO cs.AI cs.LGcited by 16

On the map: End-to-end autonomous driving → Reinforcement learning for driving

also in 1 more
Abstract (arXiv)

In the domain of autonomous driving, the offline Reinforcement Learning~(RL) approaches exhibit notable efficacy in addressing sequential decision-making problems from offline datasets. However, maintaining safety in diverse safety-critical scenarios remains a significant challenge due to long-tailed and unforeseen scenarios absent from offline datasets. In this paper, we introduce the saFety-aware strUctured Scenario representatION (FUSION), a pioneering representation learning method in offline RL to facilitate the learning of a generalizable end-to-end driving policy by leveraging structured scenario information. FUSION capitalizes on the causal relationships between the decomposed reward, cost, state, and action space, constructing a framework for structured sequential reasoning in dynamic traffic environments. We conduct extensive evaluations in two typical real-world settings of the distribution shift in autonomous vehicles, demonstrating the good balance between safety cost and utility reward compared to the current state-of-the-art safe RL and IL baselines. Empirical evidence in various driving scenarios attests that FUSION significantly enhances the safety and generalizability of autonomous driving agents, even in the face of challenging and unseen environments. Furthermore, our ablation studies reveal noticeable improvements in the integration of causal representation into the offline safe RL algorithm. Our code implementation is available at: https://sites.google.com/view/safe-fusion/.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 33 of its 43 references that are in the map. First posted on arXiv 31 Oct 2023.

Builds on

33 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing19 Apr
  2. 2022Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving24 Aug
  3. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep

Other references · 30

  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  2. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  3. 2017Constrained Policy Optimizationcited by 928
  4. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  5. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  6. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  7. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  8. 2019Batch Policy Learning under Constraintscited by 242
  9. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  10. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  11. 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
  12. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
18 more
  1. 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 100
  2. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 99
  3. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 96
  4. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  5. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  6. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
  7. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 69
  8. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 65
  9. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 60
  10. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 58
  11. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  12. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
  13. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  14. 2021Offline Reinforcement Learning with Pseudometric Learningcited by 28
  15. 2023Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlationcited by 25
  16. 2021CausalAF: Causal Autoregressive Flow for Safety-Critical Driving Scenario Generationcited by 20
  17. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  18. 2018Ensemble Bayesian Decision Making with Redundant Deep Perceptual Control Policiescited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 4; 2026: 3.

16 papers.

Sort:

By area:

  1. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  2. 2024ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularizationcited by 16
  3. 2024OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learningcited by 10
  4. 2024Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Drivingcited by 8
  5. 2023Gradient Shaping for Multi-Constraint Safe Reinforcement Learningcited by 7
  6. 2023Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approachcited by 3
  7. 2024Feasibility Consistent Representation Learning for Safe Reinforcement Learningcited by 3
  8. 2024A Safe and Efficient Self-evolving Algorithm for Decision-making and Control of Autonomous Driving Systemscited by 2
  9. 2025Offline Reinforcement Learning using Human-Aligned Reward Labeling for Autonomous Emergency Braking in Occluded Pedestrian Crossingcited by 1
  10. 2025Boundary-to-Region Supervision for Offline Safe Reinforcement Learningcited by 1
  11. 2025Online Optimization for Offline Safe Reinforcement Learningcited by 1
  12. 2026A Generalizable Physics-guided Causal Model for Trajectory Prediction in Autonomous Drivingcited by 1
4 more
  1. 2024Intrinsic Dynamics-Driven Generalizable Scene Representations for Vision-Oriented Decision-Making Applicationscited by 0
  2. 2025Generating Causal Explanations of Vehicular Agent Behavioural Interactions with Learnt Reward Profilescited by 0
  3. 2026Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiencycited by 0
  4. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platformcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving
  2. 2026DiDrive: A Risk-Aware Hierarchical Diffusion Framework for Safe Offline Reinforcement Learning in Autonomous Driving
  3. 2022Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarios
  4. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning
  5. 2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration
  6. 2022How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies
  7. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning
  8. 2025VLM-SAFE: Vision-Language Model-Guided Safety-Aware Reinforcement Learning with World Models for Autonomous Driving