mlsanityData to 14 Oct 2026

Distributional Soft Actor-Critic with Harmonic Gradient for Safe and Efficient Autonomous Driving in Multi-lane Scenarios

arXiv 2505.13532 ↗First posted 18 May 2025cs.RO cs.AI cs.LGcited by 0

On the map: End-to-end autonomous driving → Reinforcement learning for driving

also in 1 more
Abstract (arXiv)

Reinforcement learning (RL), known for its self-evolution capability, offers a promising approach to training high-level autonomous driving systems. However, handling constraints remains a significant challenge for existing RL algorithms, particularly in real-world applications. In this paper, we propose a new safety-oriented training technique called harmonic policy iteration (HPI). At each RL iteration, it first calculates two policy gradients associated with efficient driving and safety constraints, respectively. Then, a harmonic gradient is derived for policy updating, minimizing conflicts between the two gradients and consequently enabling a more balanced and stable training process. Furthermore, we adopt the state-of-the-art DSAC algorithm as the backbone and integrate it with our HPI to develop a new safe RL algorithm, DSAC-H. Extensive simulations in multi-lane scenarios demonstrate that DSAC-H achieves efficient driving performance with near-zero safety constraint violations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 13 of its 21 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

13 of 21 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin27 Nov
  2. 2022Tackling Real-World Autonomous Driving using Deep Reinforcement Learning5 Jul
  3. 2021Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-lane Scenarios12 Sep
  4. 2021Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence18 Mar

Other references · 9

  1. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  2. 2018Learning to Drive in a Daycited by 273
  3. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 134
  4. 2020Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errorscited by 85
  5. 2021Model-based Constrained Reinforcement Learning using Generalized Control Barrier Functioncited by 32
  6. 2023Distributional Soft Actor-Critic with Three Refinementscited by 25
  7. 2020Hierarchical Reinforcement Learning for Self-Driving Decision-Making without Reliance on Labeled Driving Datacited by 7
  8. 2021Self-learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersectionscited by 5
  9. 2020A Dynamic Penalty Function Approach for Constraints-Handling in Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Learning to Mastery: Achieving Safe and Efficient Real-World Autonomous Driving with Human-In-The-Loop Reinforcement Learning
  2. 2021Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning
  3. 2024Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for Discretionary Lane Change
  4. 2025Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving
  5. 2026Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency
  6. 2022SafeRL-Kit: Evaluating Efficient Reinforcement Learning Methods for Safe Autonomous Driving
  7. 2025Multi-Timescale Hierarchical Reinforcement Learning for Unified Behavior and Control of Autonomous Driving
  8. 2021Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving