mlsanityData to 14 Oct 2026

Exploiting inter-agent coupling information for efficient reinforcement learning of cooperative LQR

arXiv 2504.20927 ↗First posted 29 Apr 2025eess.SY cs.LG cs.MAcited by 1

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Developing scalable and efficient reinforcement learning algorithms for cooperative multi-agent control has received significant attention over the past years. Existing literature has proposed inexact decompositions of local Q-functions based on empirical information structures between the agents. In this paper, we exploit inter-agent coupling information and propose a systematic approach to exactly decompose the local Q-function of each agent. We develop an approximate least square policy iteration algorithm based on the proposed decomposition and identify two architectures to learn the local Q-function for each agent. We establish that the worst-case sample complexity of the decomposition is equal to the centralized case and derive necessary and sufficient graphical conditions on the inter-agent couplings to achieve better sample efficiency. We demonstrate the improved sample efficiency and computational efficiency on numerical examples.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 42 references that are in the map. First posted on arXiv 29 Apr 2025.

Builds on

21 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Scalable Reinforcement Learning for Linear-Quadratic Control of Networks29 Jan

Other references · 20

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  2. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  3. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  4. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  5. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  6. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 194
  7. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  8. 2019Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approachcited by 38
  9. 2019Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulatorcited by 23
  10. 2020F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learningcited by 10
  11. 2020Structured Policy Iteration for Linear Quadratic Regulatorcited by 8
  12. 2021On the Sample Complexity of Decentralized Linear Quadratic Regulator with Partially Nested Information Structurecited by 7
8 more
  1. 2020Cooperative Multi-Agent Reinforcement Learning with Partial Observationscited by 6
  2. 2021Learning Distributed Stabilizing Controllers for Multi-Agent Systemscited by 6
  3. 2022Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functionscited by 6
  4. 2017Diff-DAC: Distributed Actor-Critic for Average Multitask Deep Reinforcement Learningcited by 5
  5. 2018Distributed Q-Learning for Dynamically Decoupled Systemscited by 5
  6. 2020Model-Free Optimal Control of Linear Multi-Agent Systems via Decomposition and Hierarchical Approximationcited by 4
  7. 2022On the Optimal Control of Network LQR with Spatially-Exponential Decaying Structurecited by 4
  8. 2023Guaranteed Stability Margins for Decentralized Linear Quadratic Regulatorscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspectivecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mixed Q-Functionals: Advancing Value-Based Methods in Cooperative MARL with Continuous Action Domains
  2. 2025Fixing Incomplete Value Function Decomposition for Multi-Agent Reinforcement Learning
  3. 2026QSIM: Mitigating Overestimation in Multi-Agent Reinforcement Learning via Action Similarity Weighted Q-Learning
  4. 2026Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learning
  5. 2024Mitigating Relative Over-Generalization in Multi-Agent Reinforcement Learning
  6. 2025High-order Interactions Modeling for Interpretable Multi-Agent Q-Learning
  7. 2026Wireless Communication Enhanced Value Decomposition for Multi-Agent Reinforcement Learning
  8. 2024Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning