Multi-Agent Probabilistic Ensembles with Trajectory Sampling for Connected Autonomous Vehicles
On the map: End-to-end autonomous driving → Cooperative and V2X driving
Abstract (arXiv)
Autonomous Vehicles (AVs) have attracted significant attention in recent years and Reinforcement Learning (RL) has shown remarkable performance in improving the autonomy of vehicles. In that regard, the widely adopted Model-Free RL (MFRL) promises to solve decision-making tasks in connected AVs (CAVs), contingent on the readiness of a significant amount of data samples for training. Nevertheless, it might be infeasible in practice and possibly lead to learning instability. In contrast, Model-Based RL (MBRL) manifests itself in sample-efficient learning, but the asymptotic performance of MBRL might lag behind the state-of-the-art MFRL algorithms. Furthermore, most studies for CAVs are limited to the decision-making of a single AV only, thus underscoring the performance due to the absence of communications. In this study, we try to address the decision-making problem of multiple CAVs with limited communications and propose a decentralized Multi-Agent Probabilistic Ensembles with Trajectory Sampling algorithm MA-PETS. In particular, in order to better capture the uncertainty of the unknown environment, MA-PETS leverages Probabilistic Ensemble (PE) neural networks to learn from communicated samples among neighboring CAVs. Afterwards, MA-PETS capably develops Trajectory Sampling (TS)-based model-predictive control for decision-making. On this basis, we derive the multi-agent group regret bound affected by the number of agents within the communication range and mathematically validate that incorporating effective information exchange among agents into the multi-agent learning scheme contributes to reducing the group regret bound in the worst case. Finally, we empirically demonstrate the superiority of MA-PETS in terms of the sample efficiency comparable to MFBL.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving. Builds on 29 of its 62 references that are in the map. First posted on arXiv 21 Dec 2023.
Builds on
29 of 62 references.
References · 29
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
- 2017Counterfactual Multi-Agent Policy Gradientscited by 751
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
- 2018Is Q-learning Provably Efficient?cited by 487
- 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
- 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
- 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
- 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
- 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
- 2020Almost Optimal Model-Free Reinforcement Learning via Reference-Advantage Decompositioncited by 127
17 more
- 2018Efficient Exploration through Bayesian Deep Q-Networkscited by 67
- 2022A Survey on Model-based Reinforcement Learningcited by 66
- 2018Deep Reinforcement Learning for Traffic Light Control in Vehicular Networkscited by 52
- 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Drivingcited by 47
- 2019Bootstrapping Upper Confidence Boundcited by 36
- 2019Variational Regret Bounds for Reinforcement Learningcited by 32
- 2021Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Drivingcited by 22
- 2023Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospectscited by 18
- 2013Online Regret Bounds for Undiscounted Continuous Reinforcement Learningcited by 17
- 2021Autonomous Driving Strategies at Intersections: Scenarios, State-of-the-Art, and Future Outlookscited by 15
- 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14
- 2023Stochastic Graph Neural Network-based Value Decomposition for MARL in Internet of Vehiclescited by 7
- 2021Provably Efficient Multi-Agent Reinforcement Learning with Fully Decentralized Communicationcited by 5
- 2014Selecting Near-Optimal Approximate State Representations in Reinforcement Learningcited by 4
- 2022Coordination for Connected and Automated Vehicles at Non-signalized Intersections: A Value Decomposition-based Multiagent Deep Reinforcement Learning Approachcited by 4
- 2022Communication-Efficient Consensus Mechanism for Federated Reinforcement Learningcited by 2
- 2022Transfer in Reinforcement Learning via Regret Bounds for Learning Agentscited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
- 2024OPTIMA: Optimized Policy for Intelligent Multi-Agent Systems Enables Coordination-Aware Autonomous Vehicles
- 2025Goal-Oriented Multi-Agent Reinforcement Learning for Decentralized Agent Teams
- 2021Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic
- 2025Learning to Sample: Reinforcement Learning-Guided Sampling for Autonomous Vehicle Motion Planning
- 2025Impact of Collective Behaviors of Autonomous Vehicles on Urban Traffic Dynamics: A Multi-Agent Reinforcement Learning Approach
- 2024SigmaRL: A Sample-Efficient and Generalizable Multi-Agent Reinforcement Learning Framework for Motion Planning
- 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Survey