# Improve Generalization of Driving Policy at Signalized Intersections with Adversarial Learning

arXiv: https://arxiv.org/abs/2204.04403 · HTML: https://mlsanity.com/paper/2204.04403/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 9 Apr 2022.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Improved Techniques for Training Single-Image GANs](https://mlsanity.com/paper/2003.11512/) (2020)
- [An Analysis of Categorical Distributional Reinforcement Learning](https://mlsanity.com/paper/1802.08163/) (2018)
- [Risk Averse Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1904.00511/) (2019)
- [A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers](https://mlsanity.com/paper/1804.07871/) (2018)
- [Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimization](https://mlsanity.com/paper/2003.01303/) (2020)
- [Model-based Constrained Reinforcement Learning using Generalized Control Barrier Function](https://mlsanity.com/paper/2103.01556/) (2021)
- [End-to-End Deep Reinforcement Learning for Lane Keeping Assist](https://mlsanity.com/paper/1612.04340/) (2016)
- [Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence](https://mlsanity.com/paper/2103.10290/) (2021)
- [Navigating Occluded Intersections with Autonomous Vehicles using Deep Reinforcement Learning](https://mlsanity.com/paper/1705.01196/) (2017)
- [Numerically Stable Dynamic Bicycle Model for Discrete-time Control](https://mlsanity.com/paper/2011.09612/) (2020)
- [Adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraints](https://mlsanity.com/paper/1911.11397/) (2019)
- [Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Critic](https://mlsanity.com/paper/2002.05502/) (2020)
- [Model-Based Actor-Critic with Chance Constraint for Stochastic System](https://mlsanity.com/paper/2012.10716/) (2020)
- [Mixed Policy Gradient: off-policy reinforcement learning driven jointly by data and model](https://mlsanity.com/paper/2102.11513/) (2021)
- [Mixed platoon control of automated and human-driven vehicles at a signalized intersection: dynamical analysis and optimal control](https://mlsanity.com/paper/2010.16105/) (2020)
- [Self-learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections](https://mlsanity.com/paper/2110.12359/) (2021)
- [Integrated Decision and Control at Multi-Lane Intersections with Mixed Traffic Flow](https://mlsanity.com/paper/2108.13038/) (2021)

## Built on by (most cited first, 2 of 2)

- [Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersection](https://mlsanity.com/paper/2503.19690/) (2025)
- [Smoothing Policy Iteration for Zero-sum Markov Games](https://mlsanity.com/paper/2212.01623/) (2022)

## Abstract (arXiv)

Intersections are quite challenging among various driving scenes wherein the interaction of signal lights and distinct traffic actors poses great difficulty to learn a wise and robust driving policy. Current research rarely considers the diversity of intersections and stochastic behaviors of traffic participants. For practical applications, the randomness usually leads to some devastating events, which should be the focus of autonomous driving. This paper introduces an adversarial learning paradigm to boost the intelligence and robustness of driving policy for signalized intersections with dense traffic flow. Firstly, we design a static path planner which is capable of generating trackable candidate paths for multiple intersections with diversified topology. Next, a constrained optimal control problem (COCP) is built based on these candidate paths wherein the bounded uncertainty of dynamic models is considered to capture the randomness of driving environment. We propose adversarial policy gradient (APG) to solve the COCP wherein the adversarial policy is introduced to provide disturbances by seeking the most severe uncertainty while the driving policy learns to handle this situation by competition. Finally, a comprehensive system is established to conduct training and testing wherein the perception module is introduced and the human experience is incorporated to solve the yellow light dilemma. Experiments indicate that the trained policy can handle the signal lights flexibly meanwhile realizing the smooth and efficient passing with a humanoid paradigm. Besides, APG enables a large-margin improvement of the resistance to the abnormal behaviors and thus ensures a high safety level for the autonomous vehicle.