# Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic arXiv: https://arxiv.org/abs/2107.05664 ยท HTML: https://mlsanity.com/paper/2107.05664/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving. - Builds on 21 of its 38 references that are in the map. - First posted on arXiv 12 Jul 2021. ## Placed in - End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/) ## Builds on (most cited first, 21 of 21) - [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017) - [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017) - [Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/1703.06182/) (2017) - [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020) - [Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning](https://mlsanity.com/paper/1911.04175/) (2019) - [Efficient Model Learning for Human-Robot Collaborative Tasks](https://mlsanity.com/paper/1405.6341/) (2014) - [Social Coordination and Altruism in Autonomous Driving](https://mlsanity.com/paper/2107.00200/) (2021) - [Cooperative Autonomous Vehicles that Sympathize with Human Drivers](https://mlsanity.com/paper/2107.00898/) (2021) - [Deep Local Trajectory Replanning and Control for Robot Navigation](https://mlsanity.com/paper/1905.05279/) (2019) - [Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks](https://mlsanity.com/paper/1904.04375/) (2019) - [Learning How to Dynamically Route Autonomous Vehicles on Shared Roads](https://mlsanity.com/paper/1909.03664/) (2019) - [Approximate Robust Control of Uncertain Dynamical Systems](https://mlsanity.com/paper/1903.00220/) (2019) - [Spatio-temporal Dynamics of Cellular V2X Communication in Dense Vehicular Networks](https://mlsanity.com/paper/1906.08634/) (2019) - [A Maneuver-based Urban Driving Dataset and Model for Cooperative Vehicle Applications](https://mlsanity.com/paper/2005.13781/) (2020) - [Multiple Access in Cellular V2X: Performance Analysis in Highly Congested Vehicular Networks](https://mlsanity.com/paper/1809.02678/) (2018) - [Real-Time Hardware-In-the-Loop Emulation Framework for DSRC-based Connected Vehicle Applications](https://mlsanity.com/paper/1905.09267/) (2019) - [V2X System Architecture Utilizing Hybrid Gaussian Process-based Model Structures](https://mlsanity.com/paper/1903.01576/) (2019) - [A Stochastic Hybrid Framework for Driver Behavior Modeling Based on Hierarchical Dirichlet Process](https://mlsanity.com/paper/1711.05655/) (2017) - [A Driver Behavior Modeling Structure Based on Non-parametric Bayesian Stochastic Hybrid Architecture](https://mlsanity.com/paper/1807.03919/) (2018) - [Analysis of Distributed Congestion Control in Cellular Vehicle-to-everything Networks](https://mlsanity.com/paper/1904.00071/) (2019) - [Performance Analysis of Cellular-V2X with Adaptive and Selective Power Control](https://mlsanity.com/paper/2008.03453/) (2020) ## Built on by (most cited first, 11 of 11) - [Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic](https://mlsanity.com/paper/2202.00881/) (2022) - [Prediction-aware and Reinforcement Learning based Altruistic Cooperative Driving](https://mlsanity.com/paper/2211.10585/) (2022) - [Offline Reinforcement Learning for Road Traffic Control](https://mlsanity.com/paper/2201.02381/) (2022) - [Towards Learning Generalizable Driving Policies from Restricted Latent Representations](https://mlsanity.com/paper/2111.03688/) (2021) - [Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Survey](https://mlsanity.com/paper/2508.20315/) (2025) - [Active Altruism Learning and Information Sufficiency for Autonomous Driving](https://mlsanity.com/paper/2110.04580/) (2021) - [Learning-based social coordination to improve safety and robustness of cooperative autonomous vehicles in mixed traffic](https://mlsanity.com/paper/2211.11963/) (2022) - [Learning in Cooperative Multiagent Systems Using Cognitive and Machine Models](https://mlsanity.com/paper/2308.09219/) (2023) - [A Two-stage Based Social Preference Recognition in Multi-Agent Autonomous Driving System](https://mlsanity.com/paper/2310.03303/) (2023) - [Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flow](https://mlsanity.com/paper/2304.12821/) (2023) - [Towards Developing Socially Compliant Automated Vehicles: Advances, Expert Insights, and A Conceptual Framework](https://mlsanity.com/paper/2501.06089/) (2025) ## Abstract (arXiv) With the adoption of autonomous vehicles on our roads, we will witness a mixed-autonomy environment where autonomous and human-driven vehicles must learn to co-exist by sharing the same road infrastructure. To attain socially-desirable behaviors, autonomous vehicles must be instructed to consider the utility of other vehicles around them in their decision-making process. Particularly, we study the maneuver planning problem for autonomous vehicles and investigate how a decentralized reward structure can induce altruism in their behavior and incentivize them to account for the interest of other autonomous and human-driven vehicles. This is a challenging problem due to the ambiguity of a human driver's willingness to cooperate with an autonomous vehicle. Thus, in contrast with the existing works which rely on behavior models of human drivers, we take an end-to-end approach and let the autonomous agents to implicitly learn the decision-making process of human drivers only from experience. We introduce a multi-agent variant of the synchronous Advantage Actor-Critic (A2C) algorithm and train agents that coordinate with each other and can affect the behavior of human drivers to improve traffic flow and safety.