# Exploiting inter-agent coupling information for efficient reinforcement learning of cooperative LQR

arXiv: https://arxiv.org/abs/2504.20927 · HTML: https://mlsanity.com/paper/2504.20927/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multi-agent reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 29 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [Multi-agent reinforcement learning](https://mlsanity.com/deeprl/multi-agent-reinforcement-learning/)

## Builds on (most cited first, 21 of 21)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1905.05408/) (2019)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approach](https://mlsanity.com/paper/1912.09135/) (2019)
- [Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulator](https://mlsanity.com/paper/1905.12842/) (2019)
- [F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2004.11145/) (2020)
- [Structured Policy Iteration for Linear Quadratic Regulator](https://mlsanity.com/paper/2007.06202/) (2020)
- [On the Sample Complexity of Decentralized Linear Quadratic Regulator with Partially Nested Information Structure](https://mlsanity.com/paper/2110.07112/) (2021)
- [Cooperative Multi-Agent Reinforcement Learning with Partial Observations](https://mlsanity.com/paper/2006.10822/) (2020)
- [Learning Distributed Stabilizing Controllers for Multi-Agent Systems](https://mlsanity.com/paper/2103.04480/) (2021)
- [Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functions](https://mlsanity.com/paper/2202.13046/) (2022)
- [Diff-DAC: Distributed Actor-Critic for Average Multitask Deep Reinforcement Learning](https://mlsanity.com/paper/1710.10363/) (2017)
- [Distributed Q-Learning for Dynamically Decoupled Systems](https://mlsanity.com/paper/1809.08745/) (2018)
- [Model-Free Optimal Control of Linear Multi-Agent Systems via Decomposition and Hierarchical Approximation](https://mlsanity.com/paper/2008.06604/) (2020)
- [On the Optimal Control of Network LQR with Spatially-Exponential Decaying Structure](https://mlsanity.com/paper/2209.14376/) (2022)
- [Scalable Reinforcement Learning for Linear-Quadratic Control of Networks](https://mlsanity.com/paper/2401.16183/) (2024)
- [Guaranteed Stability Margins for Decentralized Linear Quadratic Regulators](https://mlsanity.com/paper/2304.07576/) (2023)

## Built on by (most cited first, 1 of 1)

- [Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspective](https://mlsanity.com/paper/2510.09937/) (2025)

## Abstract (arXiv)

Developing scalable and efficient reinforcement learning algorithms for cooperative multi-agent control has received significant attention over the past years. Existing literature has proposed inexact decompositions of local Q-functions based on empirical information structures between the agents. In this paper, we exploit inter-agent coupling information and propose a systematic approach to exactly decompose the local Q-function of each agent. We develop an approximate least square policy iteration algorithm based on the proposed decomposition and identify two architectures to learn the local Q-function for each agent. We establish that the worst-case sample complexity of the decomposition is equal to the centralized case and derive necessary and sufficient graphical conditions on the inter-agent couplings to achieve better sample efficiency. We demonstrate the improved sample efficiency and computational efficiency on numerical examples.