# Self-learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections

arXiv: https://arxiv.org/abs/2110.12359 · HTML: https://mlsanity.com/paper/2110.12359/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 24 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [SO-Net: Self-Organizing Network for Point Cloud Analysis](https://mlsanity.com/paper/1803.04249/) (2018)
- [Equivalence Between Policy Gradients and Soft Q-Learning](https://mlsanity.com/paper/1704.06440/) (2017)
- [End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances](https://mlsanity.com/paper/1911.10868/) (2019)
- [Model-free Deep Reinforcement Learning for Urban Autonomous Driving](https://mlsanity.com/paper/1904.09503/) (2019)
- [Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errors](https://mlsanity.com/paper/2001.02811/) (2020)
- [Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence](https://mlsanity.com/paper/2103.10290/) (2021)
- [Numerically Stable Dynamic Bicycle Model for Discrete-time Control](https://mlsanity.com/paper/2011.09612/) (2020)
- [Continuous Control for Automated Lane Change Behavior Based on Deep Deterministic Policy Gradient Algorithm](https://mlsanity.com/paper/1906.02275/) (2019)
- [Fixed-Dimensional and Permutation Invariant State Representation of Autonomous Driving](https://mlsanity.com/paper/2105.11299/) (2021)
- [Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-lane Scenarios](https://mlsanity.com/paper/2109.05540/) (2021)
- [Integrated Decision and Control at Multi-Lane Intersections with Mixed Traffic Flow](https://mlsanity.com/paper/2108.13038/) (2021)

## Built on by (most cited first, 5 of 5)

- [LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving](https://mlsanity.com/paper/2310.03026/) (2023)
- [Improve Generalization of Driving Policy at Signalized Intersections with Adversarial Learning](https://mlsanity.com/paper/2204.04403/) (2022)
- [A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective](https://mlsanity.com/paper/2503.23650/) (2025)
- [Predictive Lagrangian Optimization for Constrained Reinforcement Learning](https://mlsanity.com/paper/2501.15217/) (2025)
- [Distributional Soft Actor-Critic with Harmonic Gradient for Safe and Efficient Autonomous Driving in Multi-lane Scenarios](https://mlsanity.com/paper/2505.13532/) (2025)

## Abstract (arXiv)

Intersection is one of the most complex and accident-prone urban scenarios for autonomous driving wherein making safe and computationally efficient decisions is non-trivial. Current research mainly focuses on the simplified traffic conditions while ignoring the existence of mixed traffic flows, i.e., vehicles, cyclists and pedestrians. For urban roads, different participants leads to a quite dynamic and complex interaction, posing great difficulty to learn an intelligent policy. This paper develops the dynamic permutation state representation in the framework of integrated decision and control (IDC) to handle signalized intersections with mixed traffic flows. Specially, this representation introduces an encoding function and summation operator to construct driving states from environmental observation, capable of dealing with different types and variant number of traffic participants. A constrained optimal control problem is built wherein the objective involves tracking performance and the constraints for different participants and signal lights are designed respectively to assure safety. We solve this problem by offline optimizing encoding function, value function and policy function, wherein the reasonable state representation will be given by the encoding function and then served as the input of policy and value function. An off-policy training is designed to reuse observations from driving environment and backpropagation through time is utilized to update the policy function and encoding function jointly. Verification result shows that the dynamic permutation state representation can enhance the driving performance of IDC, including comfort, decision compliance and safety with a large margin. The trained driving policy can realize efficient and smooth passing in the complex intersection, guaranteeing driving intelligence and safety simultaneously.