# Topology Enhanced MARL for Multi-Agent Cooperative Decision-Making of CAVs

arXiv: https://arxiv.org/abs/2507.12110 · HTML: https://mlsanity.com/paper/2507.12110/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 68 references that are in the map.
- First posted on arXiv 16 Jul 2025.

## Builds on (most cited first, 30 of 38)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning](https://mlsanity.com/paper/1611.04717/) (2016)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [Motif: Intrinsic Motivation from Artificial Intelligence Feedback](https://mlsanity.com/paper/2310.00166/) (2023)
- [Celebrating Diversity in Shared Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.02195/) (2021)
- [Multi-Agent Game Abstraction via Graph Attention Neural Network](https://mlsanity.com/paper/1911.10715/) (2019)
- [Flow: A Modular Learning Framework for Mixed Autonomy Traffic](https://mlsanity.com/paper/1710.05465/) (2017)
- [Compressing Neural Networks using the Variational Information Bottleneck](https://mlsanity.com/paper/1802.10399/) (2018)
- [Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrifices](https://mlsanity.com/paper/2008.02790/) (2020)
- [Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain](https://mlsanity.com/paper/2109.06668/) (2021)
- [Cooperative Exploration for Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2107.11444/) (2021)
- [Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Exploration](https://mlsanity.com/paper/2111.11032/) (2021)
- [MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffer](https://mlsanity.com/paper/2206.10607/) (2022)
- [DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneck](https://mlsanity.com/paper/2102.13268/) (2021)
- [Influencing Towards Stable Multi-Agent Interactions](https://mlsanity.com/paper/2110.08229/) (2021)
- [Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning](https://mlsanity.com/paper/2408.03029/) (2024)
- [Episodic Novelty Through Temporal Distance](https://mlsanity.com/paper/2501.15418/) (2025)
- [Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL](https://mlsanity.com/paper/2209.09845/) (2022)
- [Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning](https://mlsanity.com/paper/2301.10886/) (2023)
- [ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Reward](https://mlsanity.com/paper/2210.04365/) (2022)
- [MAG-GNN: Reinforcement Learning Boosted Graph Neural Network](https://mlsanity.com/paper/2310.19142/) (2023)
- [Attention-Guided Contrastive Role Representations for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2312.04819/) (2023)
- [MAGNNETO: A Graph Neural Network-based Multi-Agent system for Traffic Engineering](https://mlsanity.com/paper/2303.18157/) (2023)
- [Learning Multi-Agent Communication with Contrastive Learning](https://mlsanity.com/paper/2307.01403/) (2023)
- [Task Placement and Resource Allocation for Edge Machine Learning: A GNN-based Multi-Agent Reinforcement Learning Paradigm](https://mlsanity.com/paper/2302.00571/) (2023)
- [Improving Intrinsic Exploration by Creating Stationary Objectives](https://mlsanity.com/paper/2310.18144/) (2023)
- [MAexp: A Generic Platform for RL-based Multi-Agent Exploration](https://mlsanity.com/paper/2404.12824/) (2024)
- [Successor-Predecessor Intrinsic Exploration](https://mlsanity.com/paper/2305.15277/) (2023)

## Built on by (most cited first, 1 of 1)

- [Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Quality](https://mlsanity.com/paper/2608.01133/) (2026)

## Abstract (arXiv)

Decentralized multi-agent cooperative decision-making in continuous environments is fundamentally bottlenecked by the curse of dimensionality, where undirected exploration typically converges to conservative local optima. We propose Topology-Enhanced Multi-Agent Reinforcement Learning (TPE-MARL) to reformulate multi-agent exploration as a structured topological traversal. We introduce the Game Topology Tensor, utilizing locality-sensitive hashing to project the continuous physical manifold into a discrete quotient space. This abstraction operates as an adversarial Information Bottleneck, decoupling strategic coordination intents from environmental noise. Within this space, a dual intrinsic reward mechanism drives exploration: a novelty reward maximizes the marginal entropy of visited topologies, while a collaboration reward, optimized via a variational Evidence Lower Bound (ELBO), minimizes conditional entropy to exploit cooperative joint configurations. Evaluations demonstrate that TPE-MARL achieves near-optimal decision distributions, closely approximating the theoretical bounds established by a Monte Carlo Tree Search (MCTS) oracle. Furthermore, physical testbed experiments validate the framework's zero-shot out-of-distribution (OOD) generalization. Supported by a spatial relaxation mechanism, the learned representations reliably execute dynamic negotiations, such as cooperative zipper-merging, exhibiting inherent robustness against real-world covariate shifts and actuation latencies.