mlsanityData to 7 Oct 2026

Distributed Policy Gradient for Linear Quadratic Networked Control with Limited Communication Range

arXiv 2403.03055 ↗First posted 5 Mar 2024cs.MA cs.LG cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper proposes a scalable distributed policy gradient method and proves its convergence to near-optimal solution in multi-agent linear quadratic networked systems. The agents engage within a specified network under local communication constraints, implying that each agent can only exchange information with a limited number of neighboring agents. On the underlying graph of the network, each agent implements its control input depending on its nearby neighbors' states in the linear quadratic control setting. We show that it is possible to approximate the exact gradient only using local information. Compared with the centralized optimal controller, the performance gap decreases to zero exponentially as the communication and control ranges increase. We also demonstrate how increasing the communication range enhances system stability in the gradient descent process, thereby elucidating a critical trade-off. The simulation results verify our theoretical findings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 50 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

22 of 50 references.

References · 22

  1. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  2. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 225
  3. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 211
  4. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  5. 2020Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularizationcited by 170
  6. 2018Learning Without Mixing: Towards A Sharp Analysis of Linear System Identificationcited by 158
  7. 2017On the Sample Complexity of the Linear Quadratic Regulatorcited by 151
  8. 2020Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Rewardcited by 46
  9. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 40
  10. 2019Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Gamescited by 38
  11. 2019Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approachcited by 33
  12. 2019LQR through the Lens of First Order Methods: Discrete-time Casecited by 30
10 more
  1. 2017Least-Squares Temporal Difference Learning for the Linear Quadratic Regulatorcited by 27
  2. 2014Multi-Agent Distributed Optimization via Inexact Consensus ADMMcited by 21
  3. 2022Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learningcited by 18
  4. 2019Exploiting Fast Decaying and Locality in Multi-Agent MDP with Tree Dependence Structurecited by 8
  5. 2021Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learningcited by 7
  6. 2022Global Convergence of Direct Policy Search for State-Feedback H_∞ Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferentialcited by 6
  7. 2020On the Convergence of Nested Decentralized Gradient Methods with Multiple Consensus and Gradient Stepscited by 5
  8. 2021Dimension-Free Rates for Natural Policy Gradient in Multi-Agent Reinforcement Learningcited by 4
  9. 2022On the Optimal Control of Network LQR with Spatially-Exponential Decaying Structurecited by 3
  10. 2022Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part Icited by 3

Built on by

Not cited yet.