mlsanityData to 7 Oct 2026

Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic

arXiv 2107.05664 ↗First posted 12 Jul 2021cs.RO cs.AIcited by 11

On the map: End-to-end autonomous driving → Cooperative and V2X driving

Abstract (arXiv)

With the adoption of autonomous vehicles on our roads, we will witness a mixed-autonomy environment where autonomous and human-driven vehicles must learn to co-exist by sharing the same road infrastructure. To attain socially-desirable behaviors, autonomous vehicles must be instructed to consider the utility of other vehicles around them in their decision-making process. Particularly, we study the maneuver planning problem for autonomous vehicles and investigate how a decentralized reward structure can induce altruism in their behavior and incentivize them to account for the interest of other autonomous and human-driven vehicles. This is a challenging problem due to the ambiguity of a human driver's willingness to cooperate with an autonomous vehicle. Thus, in contrast with the existing works which rely on behavior models of human drivers, we take an end-to-end approach and let the autonomous agents to implicitly learn the decision-making process of human drivers only from experience. We introduce a multi-agent variant of the synchronous Advantage Actor-Critic (A2C) algorithm and train agents that coordinate with each other and can affect the behavior of human drivers to improve traffic flow and safety.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving. Builds on 21 of its 38 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

21 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Social Coordination and Altruism in Autonomous Driving1 Jul

Other references · 20

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  3. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
  4. 2020Learning Latent Representations to Influence Multi-Agent Interactioncited by 92
  5. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  6. 2014Efficient Model Learning for Human-Robot Collaborative Taskscited by 35
  7. 2021Cooperative Autonomous Vehicles that Sympathize with Human Driverscited by 17
  8. 2019Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networkscited by 15
  9. 2019Deep Local Trajectory Replanning and Control for Robot Navigationcited by 15
  10. 2019Learning How to Dynamically Route Autonomous Vehicles on Shared Roadscited by 12
  11. 2019Approximate Robust Control of Uncertain Dynamical Systemscited by 9
  12. 2019Spatio-temporal Dynamics of Cellular V2X Communication in Dense Vehicular Networkscited by 8
8 more
  1. 2020A Maneuver-based Urban Driving Dataset and Model for Cooperative Vehicle Applicationscited by 8
  2. 2018Multiple Access in Cellular V2X: Performance Analysis in Highly Congested Vehicular Networkscited by 7
  3. 2019Real-Time Hardware-In-the-Loop Emulation Framework for DSRC-based Connected Vehicle Applicationscited by 7
  4. 2019V2X System Architecture Utilizing Hybrid Gaussian Process-based Model Structurescited by 6
  5. 2017A Stochastic Hybrid Framework for Driver Behavior Modeling Based on Hierarchical Dirichlet Processcited by 4
  6. 2018A Driver Behavior Modeling Structure Based on Non-parametric Bayesian Stochastic Hybrid Architecturecited by 4
  7. 2019Analysis of Distributed Congestion Control in Cellular Vehicle-to-everything Networkscited by 4
  8. 2020Performance Analysis of Cellular-V2X with Adaptive and Selective Power Controlcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 4; 2023: 3; 2024: 0; 2025: 2.

11 papers.

  1. 2022Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Trafficcited by 15
  2. 2022Prediction-aware and Reinforcement Learning based Altruistic Cooperative Drivingcited by 6
  3. 2022Offline Reinforcement Learning for Road Traffic Controlcited by 5
  4. 2021Towards Learning Generalizable Driving Policies from Restricted Latent Representationscited by 4
  5. 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Surveycited by 2
  6. 2021Active Altruism Learning and Information Sufficiency for Autonomous Drivingcited by 1
  7. 2022Learning-based social coordination to improve safety and robustness of cooperative autonomous vehicles in mixed trafficcited by 1
  8. 2023Learning in Cooperative Multiagent Systems Using Cognitive and Machine Modelscited by 1
  9. 2023A Two-stage Based Social Preference Recognition in Multi-Agent Autonomous Driving Systemcited by 1
  10. 2023Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flowcited by 0
  11. 2025Towards Developing Socially Compliant Automated Vehicles: Advances, Expert Insights, and A Conceptual Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Impact of Collective Behaviors of Autonomous Vehicles on Urban Traffic Dynamics: A Multi-Agent Reinforcement Learning Approach
  2. 2023Multi-Agent Deep Reinforcement Learning for Cooperative and Competitive Autonomous Vehicles using AutoDRIVE Ecosystem
  3. 2021Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarios
  4. 2021Linear Differential Games for Cooperative Behavior Planning of Autonomous Vehicles Using Mixed-Integer Programming
  5. 2025A Human-Oriented Cooperative Driving Approach: Integrating Driving Intention, State, and Conflict
  6. 2024Agent-Agnostic Centralized Training for Decentralized Multi-Agent Cooperative Driving
  7. 2021Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic
  8. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey