# A Two-stage Based Social Preference Recognition in Multi-Agent Autonomous Driving System

arXiv: https://arxiv.org/abs/2310.03303 · HTML: https://mlsanity.com/paper/2310.03303/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021)
- [Flow: A Modular Learning Framework for Mixed Autonomy Traffic](https://mlsanity.com/paper/1710.05465/) (2017)
- [Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning](https://mlsanity.com/paper/1911.04175/) (2019)
- [Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning](https://mlsanity.com/paper/2010.03118/) (2020)
- [Driving in Dense Traffic with Model-Free Reinforcement Learning](https://mlsanity.com/paper/1909.06710/) (2019)
- [Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimization](https://mlsanity.com/paper/2110.13827/) (2021)
- [Social Coordination and Altruism in Autonomous Driving](https://mlsanity.com/paper/2107.00200/) (2021)
- [Interaction-aware Decision-making for Automated Vehicles using Social Value Orientation](https://mlsanity.com/paper/2207.05853/) (2022)
- [Socially-Compatible Behavior Design of Autonomous Vehicles with Verification on Real Human Data](https://mlsanity.com/paper/2010.14712/) (2020)
- [Cooperative Autonomous Vehicles that Sympathize with Human Drivers](https://mlsanity.com/paper/2107.00898/) (2021)
- [LUCIDGames: Online Unscented Inverse Dynamic Games for Adaptive Trajectory Prediction and Planning](https://mlsanity.com/paper/2011.08152/) (2020)
- [B-GAP: Behavior-Rich Simulation and Navigation for Autonomous Driving](https://mlsanity.com/paper/2011.03748/) (2020)
- [Courteous Autonomous Cars](https://mlsanity.com/paper/1808.02633/) (2018)
- [Efficient Game-Theoretic Planning with Prediction Heuristic for Socially-Compliant Autonomous Driving](https://mlsanity.com/paper/2207.03673/) (2022)
- [Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic](https://mlsanity.com/paper/2107.05664/) (2021)
- [Online Parameter Estimation for Human Driver Behavior Prediction](https://mlsanity.com/paper/2005.02597/) (2020)
- [On Social Interactions of Merging Behaviors at Highway On-Ramps in Congested Traffic](https://mlsanity.com/paper/2008.06156/) (2020)

## Built on by (most cited first, 1 of 1)

- [A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives](https://mlsanity.com/paper/2503.13415/) (2025)

## Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) has become a promising solution for constructing a multi-agent autonomous driving system (MADS) in complex and dense scenarios. But most methods consider agents acting selfishly, which leads to conflict behaviors. Some existing works incorporate the concept of social value orientation (SVO) to promote coordination, but they lack the knowledge of other agents' SVOs, resulting in conservative maneuvers. In this paper, we aim to tackle the mentioned problem by enabling the agents to understand other agents' SVOs. To accomplish this, we propose a two-stage system framework. Firstly, we train a policy by allowing the agents to share their ground truth SVOs to establish a coordinated traffic flow. Secondly, we develop a recognition network that estimates agents' SVOs and integrates it with the policy trained in the first stage. Experiments demonstrate that our developed method significantly improves the performance of the driving policy in MADS compared to two state-of-the-art MARL algorithms.