# Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarios

arXiv: https://arxiv.org/abs/2103.12070 · HTML: https://mlsanity.com/paper/2103.12070/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 15 of 15)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [Soft Actor-Critic for Discrete Action Settings](https://mlsanity.com/paper/1910.07207/) (2019)
- [Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1702.08887/) (2017)
- [Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1906.04737/) (2019)
- [CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1809.05188/) (2018)
- [Multiagent Soft Q-Learning](https://mlsanity.com/paper/1804.09817/) (2018)
- [Towards Learning Multi-agent Negotiations via Self-Play](https://mlsanity.com/paper/2001.10208/) (2020)
- [Interaction-aware Decision Making with Adaptive Strategies under Merging Scenarios](https://mlsanity.com/paper/1904.06025/) (2019)
- [Microscopic Traffic Simulation by Cooperative Multi-agent Deep Reinforcement Learning](https://mlsanity.com/paper/1903.01365/) (2019)

## Built on by (most cited first, 3 of 3)

- [Learning to Model Diverse Driving Behaviors in Highly Interactive Autonomous Driving Scenarios with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.13481/) (2024)
- [Safer Autonomous Driving in a Stochastic, Partially-Observable Environment by Hierarchical Contingency Planning](https://mlsanity.com/paper/2204.06509/) (2022)
- [Emergency Lane-Change Simulation: A Behavioral Guidance Approach for Risky Scenario Generation](https://mlsanity.com/paper/2603.20234/) (2026)

## Abstract (arXiv)

Recently, autonomous driving has made substantial progress in addressing the most common traffic scenarios like intersection navigation and lane changing. However, most of these successes have been limited to scenarios with well-defined traffic rules and require minimal negotiation with other vehicles. In this paper, we introduce a previously unconsidered, yet everyday, high-conflict driving scenario requiring negotiations between agents of equal rights and priorities. There exists no centralized control structure and we do not allow communications. Therefore, it is unknown if other drivers are willing to cooperate, and if so to what extent. We train policies to robustly negotiate with opposing vehicles of an unobservable degree of cooperativeness using multi-agent reinforcement learning (MARL). We propose Discrete Asymmetric Soft Actor-Critic (DASAC), a maximum-entropy off-policy MARL algorithm allowing for centralized training with decentralized execution. We show that using DASAC we are able to successfully negotiate and traverse the scenario considered over 99% of the time. Our agents are robust to an unknown timing of opponent decisions, an unobservable degree of cooperativeness of the opposing vehicle, and previously unencountered policies. Furthermore, they learn to exhibit human-like behaviors such as defensive driving, anticipating solution options and interpreting the behavior of other agents.