mlsanityData to 7 Oct 2026

A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving

arXiv 2405.01440 ↗First posted 12 Apr 2024cs.RO cs.AI cs.LGcited by 17

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Reinforcement learning has emerged as an important approach for autonomous driving. A reward function is used in reinforcement learning to establish the learned skill objectives and guide the agent toward the optimal policy. Since autonomous driving is a complex domain with partly conflicting objectives with varying degrees of priority, developing a suitable reward function represents a fundamental challenge. This paper aims to highlight the gap in such function design by assessing different proposed formulations in the literature and dividing individual objectives into Safety, Comfort, Progress, and Traffic Rules compliance categories. Additionally, the limitations of the reviewed reward functions are discussed, such as objectives aggregation and indifference to driving context. Furthermore, the reward categories are frequently inadequately formulated and lack standardization. This paper concludes by proposing future research that potentially addresses the observed shortcomings in rewards, including a reward validation framework and structured rewards that are context-aware and able to resolve conflicts.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 42 of its 69 references that are in the map. First posted on arXiv 12 Apr 2024.

Builds on

42 of 69 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving5 Jun
  2. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors8 May
  3. 2021Evaluating the Robustness of Deep Reinforcement Learning for Autonomous Policies in a Multi-agent Urban Driving Environment22 Dec
  4. 2021Reward (Mis)design for Autonomous Driving28 Apr

Other references · 38

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  3. 2018An Introduction to Deep Reinforcement Learningcited by 201
  4. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  5. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  6. 2017On a Formal Model of Safe and Scalable Self-driving Carscited by 129
  7. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 99
  8. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
  9. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  10. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  11. 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 79
  12. 2020Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Methodcited by 74
26 more
  1. 2019Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Drivingcited by 56
  2. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  3. 2019Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Drivingcited by 52
  4. 2019Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehiclescited by 51
  5. 2016Learning to Perform Physics Experiments via Deep Reinforcement Learningcited by 47
  6. 2019Human-Like Autonomous Car-Following Model with Deep Reinforcement Learningcited by 47
  7. 2020Computing Systems for Autonomous Driving: State-of-the-Art and Challengescited by 47
  8. 2019Liability, Ethics, and Culture-Aware Behavior Specification using Rulebookscited by 41
  9. 2018Automated Speed and Lane Change Decision Making using Deep Reinforcement Learningcited by 36
  10. 2019Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Drivingcited by 36
  11. 2020Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Drivingcited by 34
  12. 2018Urban Driving with Multi-Objective Deep Reinforcement Learningcited by 30
  13. 2019Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Drivingcited by 30
  14. 2020Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learningcited by 30
  15. 2021Rule-based Optimal Control for Autonomous Drivingcited by 21
  16. 2017Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environmentscited by 19
  17. 2017Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Mergecited by 17
  18. 2019LeTS-Drive: Driving in a Crowd by Learning from Tree Searchcited by 13
  19. 2019Interaction-aware Decision Making with Adaptive Strategies under Merging Scenarioscited by 12
  20. 2021The Reasonable Crowd: Towards evidence-based and interpretable models of driving behaviorcited by 12
  21. 2020Reinforcement Learning with Iterative Reasoning for Merging in Dense Trafficcited by 10
  22. 2019Quadratic Q-network for Learning Continuous Control for Autonomous Vehiclescited by 8
  23. 2020An End-to-end Deep Reinforcement Learning Approach for the Long-term Short-term Planning on the Frenet Spacecited by 5
  24. 2023Interpretable Reinforcement Learning for Robotics and Continuous Controlcited by 5
  25. 2020Safe Trajectory Planning Using Reinforcement Learning for Self Drivingcited by 2
  26. 2022Rationale-aware Autonomous Driving Policy utilizing Safety Force Field implemented on CARLA Simulatorcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 9; 2026: 4.

17 papers.

Sort:

By area:

  1. 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Drivingcited by 49
  2. 2024ELEMENTAL: Interactive Learning from Demonstrations and Vision-Language Models for Reward Design in Roboticscited by 8
  3. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Drivingcited by 7
  4. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulatorcited by 7
  5. 2024From Words to Wheels: Automated Style-Customized Policy Generation for Autonomous Drivingcited by 5
  6. 2025LearningFlow: Automated Policy Learning Workflow for Urban Driving with Large Language Modelscited by 4
  7. 2025A Differentiated Reward Method for Reinforcement Learning based Multi-Vehicle Cooperative Decision-Making Algorithmscited by 1
  8. 2025Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Drivingcited by 1
  9. 2024Model Checking for Reinforcement Learning in Autonomous Driving: One Can Do More Than You Think!cited by 0
  10. 2025Towards a Reward-Free Reinforcement Learning Framework for Vehicle Controlcited by 0
  11. 2025Prompting Decision Transformers for Zero-Shot Reach-Avoid Policiescited by 0
  12. 2025Diverse and Adaptive Behavior Curriculum for Autonomous Driving: A Student-Teacher Framework with Multi-Agent RLcited by 0
5 more
  1. 2025Orchestrate, Generate, Reflect: A VLM-Based Multi-Agent Collaboration Framework for Automated Driving Policy Learningcited by 0
  2. 2026Multi-Objective Reinforcement Learning for Tactical Decision Making for Trucks in Highway Trafficcited by 0
  3. 2026MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inferencecited by 0
  4. 2026Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Drivingcited by 0
  5. 2026Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Qualitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ROAD: Responsibility-Oriented Reward Design for Reinforcement Learning in Autonomous Driving
  2. 2024Generating and Evolving Reward Functions for Highway Driving with Large Language Models
  3. 2025Design of Reward Function on Reinforcement Learning for Automated Driving
  4. 2022Robust AI Driving Strategy for Autonomous Vehicles
  5. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective
  6. 2022Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving
  7. 2024Optimizing Low-Speed Autonomous Driving: A Reinforcement Learning Approach to Route Stability and Maximum Speed
  8. 2024LORD: Large Models based Opposite Reward Design for Autonomous Driving