mlsanityData to 14 Oct 2026

Multi-agent Uncertainty-Aware Pessimistic Model-Based Reinforcement Learning for Connected Autonomous Vehicles

arXiv 2503.20462 ↗First posted 26 Mar 2025cs.MAcited by 0

On the map: End-to-end autonomous driving → Cooperative and V2X driving

Abstract (arXiv)

Deep Reinforcement Learning (DRL) holds significant promise for achieving human-like Autonomous Vehicle (AV) capabilities, but suffers from low sample efficiency and challenges in reward design. Model-Based Reinforcement Learning (MBRL) offers improved sample efficiency and generalizability compared to Model-Free Reinforcement Learning (MFRL) in various multi-agent decision-making scenarios. Nevertheless, MBRL faces critical difficulties in estimating uncertainty during the model learning phase, thereby limiting its scalability and applicability in real-world scenarios. Additionally, most Connected Autonomous Vehicle (CAV) studies focus on single-agent decision-making, while existing multi-agent MBRL solutions lack computationally tractable algorithms with Probably Approximately Correct (PAC) guarantees, an essential factor for ensuring policy reliability with limited training data. To address these challenges, we propose MA-PMBRL, a novel Multi-Agent Pessimistic Model-Based Reinforcement Learning framework for CAVs, incorporating a max-min optimization approach to enhance robustness and decision-making. To mitigate the inherent subjectivity of uncertainty estimation in MBRL and avoid incurring catastrophic failures in AV, MA-PMBRL employs a pessimistic optimization framework combined with Projected Gradient Descent (PGD) for both model and policy learning. MA-PMBRL also employs general function approximations under partial dataset coverage to enhance learning efficiency and system-level performance. By bounding the suboptimality of the resulting policy under mild theoretical assumptions, we successfully establish PAC guarantees for MA-PMBRL, demonstrating that the proposed framework represents a significant step toward scalable, efficient, and reliable multi-agent decision-making for CAVs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving. Builds on 32 of its 64 references that are in the map. First posted on arXiv 26 Mar 2025.

Builds on

32 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Multi-Agent Probabilistic Ensembles with Trajectory Sampling for Connected Autonomous Vehicles21 Dec

Other references · 31

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  4. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  6. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  7. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  8. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  9. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  10. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
  11. 2021Bellman-consistent Pessimism for Offline Reinforcement Learningcited by 269
  12. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
19 more
  1. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 177
  2. 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 164
  3. 2017Non-convex Optimization for Machine Learningcited by 133
  4. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 117
  5. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  6. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 110
  7. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 96
  8. 2022A Survey on Model-based Reinforcement Learningcited by 66
  9. 2018Deep Reinforcement Learning for Traffic Light Control in Vehicular Networkscited by 52
  10. 2020Overcoming Model Bias for Robust Offline Deep Reinforcement Learningcited by 43
  11. 2020Multi-Agent Reinforcement Learning in Stochastic Networked Systemscited by 32
  12. 2021Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Drivingcited by 22
  13. 2023Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospectscited by 18
  14. 2018Learning to Share and Hide Intentions using Information Regularizationcited by 13
  15. 2023Models as Agents: Optimizing Multi-Step Predictions of Interactive Local Models in Model-Based Multi-Agent Reinforcement Learningcited by 10
  16. 2023Stochastic Graph Neural Network-based Value Decomposition for MARL in Internet of Vehiclescited by 7
  17. 2021Provably Efficient Multi-Agent Reinforcement Learning with Fully Decentralized Communicationcited by 5
  18. 2022Coordination for Connected and Automated Vehicles at Non-signalized Intersections: A Value Decomposition-based Multiagent Deep Reinforcement Learning Approachcited by 4
  19. 2022Communication-Efficient Consensus Mechanism for Federated Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving
  2. 2024Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach with Attention and Hierarchical Game Priors
  3. 2025Robust and Safe Multi-Agent Reinforcement Learning with Communication for Autonomous Vehicles: From Simulation to Hardware
  4. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
  5. 2026All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning
  6. 2025InDRiVE: Intrinsic Disagreement based Reinforcement for Vehicle Exploration through Curiosity Driven Generalized World Model
  7. 2025Unveiling Uncertainty-Aware Autonomous Cooperative Learning Based Planning Strategy
  8. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning