mlsanityData to 7 Oct 2026

Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving

arXiv 2603.20230 ↗First posted 6 Mar 2026cs.RO cs.AI cs.LGcited by 0

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Autonomous driving involves multiple, often conflicting objectives such as safety, efficiency, and comfort. In reinforcement learning (RL), these objectives are typically combined through weighted summation, which collapses their relative priorities and often yields policies that violate safety-critical constraints. To overcome this limitation, we introduce the Preordered Multi-Objective MDP (Pr-MOMDP), which augments standard MOMDPs with a preorder over reward components. This structure enables reasoning about actions with respect to a hierarchy of objectives rather than a scalar signal. To make this structure actionable, we extend distributional RL with a novel pairwise comparison metric, Quantile Dominance (QD), that evaluates action return distributions without reducing them into a single statistic. Building on QD, we propose an algorithm for extracting optimal subsets, the subset of actions that remain non-dominated under each objective, which allows precedence information to shape both decision-making and training targets. Our framework is instantiated with Implicit Quantile Networks (IQN), establishing a concrete implementation while preserving compatibility with a broad class of distributional RL methods. Experiments in Carla show improved success rates, fewer collisions and off-road events, and deliver statistically more robust policies than IQN and ensemble-IQN baselines. By ensuring policies respect rewards preorder, our work advances safer, more reliable autonomous driving systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 30 of its 40 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

30 of 40 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving10 May
  2. 2025Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Driving8 May
  3. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Driving14 Jan
  4. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions20 Sep
  5. 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving12 Apr
  6. 2021Reward (Mis)design for Autonomous Driving28 Apr

Other references · 24

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  2. 2022Planning-oriented Autonomous Drivingcited by 1,068
  3. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  5. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  6. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  7. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  8. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  9. 2022Defining and Characterizing Reward Hackingcited by 184
  10. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  11. 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
  12. 2017Hybrid Reward Architecture for Reinforcement Learningcited by 57
12 more
  1. 2019Liability, Ethics, and Culture-Aware Behavior Specification using Rulebookscited by 41
  2. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
  3. 2018Urban Driving with Multi-Objective Deep Reinforcement Learningcited by 30
  4. 2021Distributional Reinforcement Learning for Multi-Dimensional Reward Functionscited by 21
  5. 2024Foundations of Multivariate Distributional Reinforcement Learningcited by 16
  6. 2021The Reasonable Crowd: Towards evidence-based and interpretable models of driving behaviorcited by 12
  7. 2019Distributional Reward Decomposition for Reinforcement Learningcited by 11
  8. 2022Value Function Decomposition for Iterative Design of Reinforcement Learning Agentscited by 6
  9. 2022Risk-Sensitive Policy with Distributional Reinforcement Learningcited by 6
  10. 2021Navigation In Urban Environments Amongst Pedestrians Using Multi-Objective Deep Reinforcement Learningcited by 5
  11. 2025AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learningcited by 5
  12. 2024Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptionscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Multi-Objective Reinforcement Learning for Tactical Decision Making for Trucks in Highway Traffic
  2. 2021Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning
  3. 2022Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving
  4. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
  5. 2026Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving
  6. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  7. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning
  8. 2025V-Max: A Reinforcement Learning Framework for Autonomous Driving