# Urban Driving with Multi-Objective Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1811.08586 · HTML: https://mlsanity.com/paper/1811.08586/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2018.

## Built on by (most cited first, 30 of 30)

- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [Reward (Mis)design for Autonomous Driving](https://mlsanity.com/paper/2104.13906/) (2021)
- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [Thresholded Lexicographic Ordered Multiobjective Reinforcement Learning](https://mlsanity.com/paper/2408.13493/) (2024)
- [Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions](https://mlsanity.com/paper/2409.13144/) (2024)
- [Navigation In Urban Environments Amongst Pedestrians Using Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/2110.05205/) (2021)
- [Prioritized Soft Q-Decomposition for Lexicographic Reinforcement Learning](https://mlsanity.com/paper/2310.02360/) (2023)
- [Generalized Multi-Objective Reinforcement Learning with Envelope Updates in URLLC-enabled Vehicular Networks](https://mlsanity.com/paper/2405.11331/) (2024)
- [Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach](https://mlsanity.com/paper/2510.20235/) (2025)
- [MODULI: Unlocking Preference Generalization via Diffusion Models for Offline Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2408.15501/) (2024)
- [Goal-constrained Sparse Reinforcement Learning for End-to-End Driving](https://mlsanity.com/paper/2103.09189/) (2021)
- [I Know You Can't See Me: Dynamic Occlusion-Aware Safety Validation of Strategic Planners for Autonomous Vehicles Using Hypergames](https://mlsanity.com/paper/2109.09807/) (2021)
- [Generalized dynamic cognitive hierarchy models for strategic driving behavior](https://mlsanity.com/paper/2109.09861/) (2021)
- [Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach](https://mlsanity.com/paper/2302.02179/) (2023)
- [Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Driving](https://mlsanity.com/paper/2505.05223/) (2025)
- [Explanation-Aware Experience Replay in Rule-Dense Environments](https://mlsanity.com/paper/2109.14711/) (2021)
- [Autonomous and Human-Driven Vehicles Interacting in a Roundabout: A Quantitative and Qualitative Evaluation](https://mlsanity.com/paper/2309.08254/) (2023)
- [Navigating Trade-offs: Policy Summarization for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2411.04784/) (2024)
- [Intelligent Autonomous Intersection Management](https://mlsanity.com/paper/2202.04224/) (2022)
- [gTLO: A Generalized and Non-linear Multi-Objective Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2204.04988/) (2022)
- [Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving](https://mlsanity.com/paper/2505.06737/) (2025)
- [LPPG-RL: Lexicographically Projected Policy Gradient Reinforcement Learning with Subproblem Exploration](https://mlsanity.com/paper/2511.08339/) (2025)
- [A taxonomy of strategic human interactions in traffic conflicts](https://mlsanity.com/paper/2109.13367/) (2021)
- [Revealed Multi-Objective Utility Aggregation in Human Driving](https://mlsanity.com/paper/2303.07435/) (2023)
- [Effective Reward Specification in Deep Reinforcement Learning](https://mlsanity.com/paper/2412.07177/) (2024)
- [Multi-Objective Reinforcement Learning for Water Management](https://mlsanity.com/paper/2505.01094/) (2025)
- [Pareto-NRPA: A Novel Monte-Carlo Search Algorithm for Multi-Objective Optimization](https://mlsanity.com/paper/2507.19109/) (2025)
- [Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving](https://mlsanity.com/paper/2603.20230/) (2026)
- [TREX: Trajectory Explanations for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2603.21988/) (2026)

## Abstract (arXiv)

Autonomous driving is a challenging domain that entails multiple aspects: a vehicle should be able to drive to its destination as fast as possible while avoiding collision, obeying traffic rules and ensuring the comfort of passengers. In this paper, we present a deep learning variant of thresholded lexicographic Q-learning for the task of urban driving. Our multi-objective DQN agent learns to drive on multi-lane roads and intersections, yielding and changing lanes according to traffic rules. We also propose an extension for factored Markov Decision Processes to the DQN architecture that provides auxiliary features for the Q function. This is shown to significantly improve data efficiency. We then show that the learned policy is able to zero-shot transfer to a ring road without sacrificing performance.