# A Value Based Parallel Update MCTS Method for Multi-Agent Cooperative Decision Making of Connected and Automated Vehicles

arXiv: https://arxiv.org/abs/2409.13783 · HTML: https://mlsanity.com/paper/2409.13783/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 64 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 16 of 16)

- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [Monte Carlo Tree Search: A Review of Recent Modifications and Applications](https://mlsanity.com/paper/2103.04931/) (2021)
- [Flow: A Modular Learning Framework for Mixed Autonomy Traffic](https://mlsanity.com/paper/1710.05465/) (2017)
- [Cooperative Driving at Unsignalized Intersections Using Tree Search](https://mlsanity.com/paper/1902.01024/) (2019)
- [Cooperative Decision Making of Connected Automated Vehicles at Multi-lane Merging Zone: A Coalitional Game Approach](https://mlsanity.com/paper/2103.07887/) (2021)
- [A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network](https://mlsanity.com/paper/2010.05437/) (2020)
- [A Systematic Survey of Control Techniques and Applications in Connected and Automated Vehicles](https://mlsanity.com/paper/2303.05665/) (2023)
- [ T3 OMVP: A Transformer-based Time and Team Reinforcement Learning Scheme for Observation-constrained Multi-Vehicle Pursuit in Urban Area](https://mlsanity.com/paper/2203.00183/) (2022)
- [Optimization-Based Path-Planning for Connected and non-Connected Automated Vehicles](https://mlsanity.com/paper/2104.06778/) (2021)
- [A Bi-Level Cooperative Driving Strategy Allowing Lane Changes](https://mlsanity.com/paper/1912.11495/) (2019)
- [TrafficMCTS: A Closed-Loop Traffic Flow Generation Framework with Group-Based Monte Carlo Tree Search](https://mlsanity.com/paper/2308.12797/) (2023)
- [Decentralized Optimal Control of Connected Automated Vehicles at Signal-Free Intersections Including Comfort-Constrained Turns and Safety Guarantees](https://mlsanity.com/paper/1903.04600/) (2019)
- [Trajectory Planning for Connected and Automated Vehicles at Isolated Signalized Intersections under Mixed Traffic Environment](https://mlsanity.com/paper/2102.03518/) (2021)
- [Towards convergence to Nash equilibria in two-team zero-sum games](https://mlsanity.com/paper/2111.04178/) (2021)
- [Optimal trajectory planning meets network-level routing: Integrated control framework for emerging mobility systems](https://mlsanity.com/paper/2311.13193/) (2023)

## Built on by (most cited first, 2 of 2)

- [A Differentiated Reward Method for Reinforcement Learning based Multi-Vehicle Cooperative Decision-Making Algorithms](https://mlsanity.com/paper/2502.00352/) (2025)
- [Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Quality](https://mlsanity.com/paper/2608.01133/) (2026)

## Abstract (arXiv)

To solve the problem of lateral and logitudinal joint decision-making of multi-vehicle cooperative driving for connected and automated vehicles (CAVs), this paper proposes a Monte Carlo tree search (MCTS) method with parallel update for multi-agent Markov game with limited horizon and time discounted setting. By analyzing the parallel actions in the multi-vehicle joint action space in the partial-steady-state traffic flow, the parallel update method can quickly exclude potential dangerous actions, thereby increasing the search depth without sacrificing the search breadth. The proposed method is tested in a large number of randomly generated traffic flow. The experiment results show that the algorithm has good robustness and better performance than the SOTA reinforcement learning algorithms and heuristic methods. The vehicle driving strategy using the proposed algorithm shows rationality beyond human drivers, and has advantages in traffic efficiency and safety in the coordinating zone.