Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Quality
On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic
Abstract (arXiv)
Evaluating Multi-Agent Reinforcement Learning (MARL) policies in autonomous driving fundamentally relies on extrinsic statistical indicators (e.g., reward curves and success rates), which often mask intrinsic policy degradation and algorithmic blind spots. To break this black-box evaluation, this letter proposes a novel information-theoretic diagnostic framework. By leveraging a fully converged Monte Carlo Tree Search (MCTS) as an asymptotic oracle, we establish a theoretical ground-truth baseline distribution. We formulate a bounded policy optimality score (Mopt) using the forward KL divergence to rigorously penalize fatal collaborative omissions. Crucially, we semantically decouple this metric into lateral and longitudinal dimensions, creating a granular "semantic microscope". Extensive spatial and temporal diagnostics on state-of-the-art MARL architectures and exploration mechanisms demonstrate that our framework conclusively exposes hidden directional biases, identifies temporal average-policy traps, and transforms heuristic hyperparameter tuning into a visually trackable trajectory optimization. This framework establishes a rigorous, model-agnostic standard for benchmarking intrinsic multi-agent policy quality.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 36 references that are in the map. First posted on arXiv 2 Aug 2026.
Builds on
25 of 36 references.
Closest ancestors · 2
Same direction, newest first
- 2024SPformer: A Transformer Based DRL Decision Making Method for Connected Automated Vehicles23 Sep
- 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving12 Apr
Other references · 23
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015Trust Region Policy Optimizationcited by 2,580
- 2017Deep Reinforcement Learning that Matterscited by 754
- 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
- 2021Monte Carlo Tree Search: A Review of Recent Modifications and Applicationscited by 161
- 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
- 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 98
- 2024RRM: Robust Reward Model Training Mitigates Reward Hackingcited by 76
- 2021A Survey on Interpretable Reinforcement Learningcited by 73
- 2021Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Trafficcited by 41
- 2021Greedification Operators for Policy Optimization: Investigating Forward and Reverse KL Divergencescited by 41
11 more
- 2020Evaluating the Performance of Reinforcement Learning Algorithmscited by 36
- 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
- 2023Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospectscited by 18
- 2022Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarioscited by 13
- 2020A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehiclescited by 12
- 2023Successor-Predecessor Intrinsic Explorationcited by 6
- 2024Checkmating One, by Using Many: Combining Mixture of Experts with MCTS to Improve in Chesscited by 4
- 2024Effective Exploration Based on the Structural Information Principlescited by 4
- 2020Policy Evaluation and Seeking for Multi-Agent Reinforcement Learning via Best Responsecited by 2
- 2024A Value Based Parallel Update MCTS Method for Multi-Agent Cooperative Decision Making of Connected and Automated Vehiclescited by 2
- 2025Topology Enhanced MARL for Multi-Agent Cooperative Decision-Making of CAVscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
- 2026Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving
- 2026Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving
- 2025Post-Training and Test-Time Scaling of Generative Agent Behavior Models for Interactive Autonomous Driving
- 2026What Probing Reveals about Autonomous Driving: Linking Internal Prediction Errors to Ego Planning
- 2025Action Space Reduction Strategies for Reinforcement Learning in Autonomous Driving
- 2025MARL-OT: Multi-Agent Reinforcement Learning Guided Online Fuzzing to Detect Safety Violation in Autonomous Driving Systems
- 2022Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving