# High-order Interactions Modeling for Interpretable Multi-Agent Q-Learning

arXiv: https://arxiv.org/abs/2510.20218 · HTML: https://mlsanity.com/paper/2510.20218/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Builds on (most cited first, 28 of 28)

- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1905.05408/) (2019)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [QPLEX: Duplex Dueling Multi-Agent Q-Learning](https://mlsanity.com/paper/2008.01062/) (2020)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.10800/) (2020)
- [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03963/) (2019)
- [SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2212.07489/) (2022)
- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.07169/) (2020)
- [Celebrating Diversity in Shared Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.02195/) (2021)
- [Shapley Q-value: A Local Reward Approach to Solve Global Reward Games](https://mlsanity.com/paper/1907.05707/) (2019)
- [Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery](https://mlsanity.com/paper/1912.03558/) (2019)
- [Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workers](https://mlsanity.com/paper/2212.11498/) (2022)
- [SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning](https://mlsanity.com/paper/2105.15013/) (2021)
- [Action Semantics Network: Considering the Effects of Actions in Multiagent Systems](https://mlsanity.com/paper/1907.11461/) (2019)
- [Multi-Robot Cooperative Socially-Aware Navigation Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2309.15234/) (2023)
- [NA2Q: Neural Attention Additive Model for Interpretable Multi-Agent Q-Learning](https://mlsanity.com/paper/2304.13383/) (2023)
- [Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios](https://mlsanity.com/paper/2210.02300/) (2022)
- [AI for Explaining Decisions in Multi-Agent Environments](https://mlsanity.com/paper/1910.04404/) (2019)
- [MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.12449/) (2022)
- [Stable and Efficient Shapley Value-Based Reward Reallocation for Multi-Agent Reinforcement Learning of Autonomous Vehicles](https://mlsanity.com/paper/2203.06333/) (2022)
- [MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees](https://mlsanity.com/paper/2209.07225/) (2022)
- [Optimizing Interpretable Decision Tree Policies for Reinforcement Learning](https://mlsanity.com/paper/2408.11632/) (2024)
- [CoFrNets: Interpretable Neural Architecture Inspired by Continued Fractions](https://mlsanity.com/paper/2506.05586/) (2025)

## Built on by (most cited first, 2 of 2)

- [Conditional Diffusion Model for Multi-Agent Dynamic Task Decomposition](https://mlsanity.com/paper/2511.13137/) (2025)
- [Flowing Faster to Coordinate: One-Step Online Multi-Agent Flow Policies](https://mlsanity.com/paper/2610.01882/) (2026)

## Abstract (arXiv)

The ability to model interactions among agents is crucial for effective coordination and understanding their cooperation mechanisms in multi-agent reinforcement learning (MARL). However, previous efforts to model high-order interactions have been primarily hindered by the combinatorial explosion or the opaque nature of their black-box network structures. In this paper, we propose a novel value decomposition framework, called Continued Fraction Q-Learning (QCoFr), which can flexibly capture arbitrary-order agent interactions with only linear complexity Oleft(nright) in the number of agents, thus avoiding the combinatorial explosion when modeling rich cooperation. Furthermore, we introduce the variational information bottleneck to extract latent information for estimating credits. This latent information helps agents filter out noisy interactions, thereby significantly enhancing both cooperation and interpretability. Extensive experiments demonstrate that QCoFr not only consistently achieves better performance but also provides interpretability that aligns with our theoretical analysis.