mlsanityData to 7 Oct 2026

Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic

arXiv 2111.06318 ↗First posted 11 Nov 2021cs.LG cs.MAcited by 20

On the map: End-to-end autonomous driving → Cooperative and V2X driving

Abstract (arXiv)

Autonomous driving has attracted significant research interests in the past two decades as it offers many potential benefits, including releasing drivers from exhausting driving and mitigating traffic congestion, among others. Despite promising progress, lane-changing remains a great challenge for autonomous vehicles (AV), especially in mixed and dynamic traffic scenarios. Recently, reinforcement learning (RL), a powerful data-driven control method, has been widely explored for lane-changing decision makings in AVs with encouraging results demonstrated. However, the majority of those studies are focused on a single-vehicle setting, and lane-changing in the context of multiple AVs coexisting with human-driven vehicles (HDVs) have received scarce attention. In this paper, we formulate the lane-changing decision making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem, where each AV makes lane-changing decisions based on the motions of both neighboring AVs and HDVs. Specifically, a multi-agent advantage actor-critic network (MA2C) is developed with a novel local reward design and a parameter sharing scheme. In particular, a multi-objective reward function is proposed to incorporate fuel efficiency, driving comfort, and safety of autonomous driving. Comprehensive experimental results, conducted under three different traffic densities and various levels of human driver aggressiveness, show that our proposed MARL framework consistently outperforms several state-of-the-art benchmarks in terms of efficiency, safety and driver comfort.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving. Builds on 21 of its 44 references that are in the map. First posted on arXiv 11 Nov 2021.

Builds on

21 of 44 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  6. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  7. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  8. 2019Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Controlcited by 159
  9. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 119
  10. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  11. 2020Multi-agent Reinforcement Learning for Networked System Controlcited by 51
  12. 2021Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Trafficcited by 38
9 more
  1. 2018Automated Speed and Lane Change Decision Making using Deep Reinforcement Learningcited by 36
  2. 2019Autonomous Driving using Safe Reinforcement Learning by Incorporating a Regret-based Human Lane-Changing Decision Modelcited by 14
  3. 2019Continuous Control for Automated Lane Change Behavior Based on Deep Deterministic Policy Gradient Algorithmcited by 8
  4. 2020A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Networkcited by 8
  5. 2020Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestioncited by 5
  6. 2020A Cooperative Control Framework for CAV Lane Change in a Mixed Traffic Environmentcited by 4
  7. 2018Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviorscited by 2
  8. 2019Efficient Motion Planning for Automated Lane Change based on Imitation Learning and Mixed-Integer Optimizationcited by 2
  9. 2021Towards Safe Control of Continuum Manipulator Using Shielded Multiagent Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 3; 2025: 6; 2026: 5.

20 papers.

Sort:

By area:

  1. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  2. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priorscited by 15
  3. 2025AgentGit: A Version Control Framework for Reliable and Scalable LLM-Powered Multi-Agent Systemscited by 8
  4. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  5. 2025A Constrained Multi-Agent Reinforcement Learning Approach to Autonomous Traffic Signal Controlcited by 3
  6. 2026GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent Systemscited by 3
  7. 2023Hiding in Plain Sight: Differential Privacy Noise Exploitation for Evasion-resilient Localized Poisoning Attacks in Multiagent Reinforcement Learningcited by 2
  8. 2024Learning to Model Diverse Driving Behaviors in Highly Interactive Autonomous Driving Scenarios with Multi-Agent Reinforcement Learningcited by 2
  9. 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Surveycited by 2
  10. 2023BRNES: Enabling Security and Privacy-aware Experience Sharing in Multiagent Robotic and Autonomous Systemscited by 1
  11. 2024Mix Q-learning for Lane Changing: A Collaborative Decision-Making Method in Multi-Agent Deep Reinforcement Learningcited by 1
  12. 2025Safe and Efficient CAV Lane Changing using Decentralised Safety Shieldscited by 1
8 more
  1. 2026Pedestrian-Aware LLM-Driven Behavioral Planning for Autonomous Vehiclescited by 1
  2. 2022Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learningcited by 0
  3. 2023Recent Progress in Energy Management of Connected Hybrid Electric Vehicles Using Reinforcement Learningcited by 0
  4. 2025Counterfactual experience augmented off-policy reinforcement learningcited by 0
  5. 2025Multi-Agent Coordination in Autonomous Vehicle Routing: A Simulation-Based Study of Communication, Memory, and Routing Loopscited by 0
  6. 2026A Collaborative Safety Shield for Safe and Efficient CAV Lane Changes in Congested On-Ramp Mergingcited by 0
  7. 2026Episodic Memory Temporal Consistency for Cooperative Multi-Agent Reinforcement Learningcited by 0
  8. 2026Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Trafficcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Meta Reinforcement Learning-Based Lane Change Strategy for Autonomous Vehicles
  2. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
  3. 2025Impact of Collective Behaviors of Autonomous Vehicles on Urban Traffic Dynamics: A Multi-Agent Reinforcement Learning Approach
  4. 2024Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approach
  5. 2025An Intention-driven Lane Change Framework Considering Heterogeneous Dynamic Cooperation in Mixed-traffic Environment ↗
  6. 2025A Hybrid Input based Deep Reinforcement Learning for Lane Change Decision-Making of Autonomous Vehicle
  7. 2025Resolve Highway Conflict in Multi-Autonomous Vehicle Controls with Local State Attention
  8. 2021Quick Learner Automated Vehicle Adapting its Roadmanship to Varying Traffic Cultures with Meta Reinforcement Learning