mlsanityData to 7 Oct 2026

Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approach

arXiv 1912.09135 ↗First posted 19 Dec 2019eess.SY cs.LG cs.SYcited by 33

Not on the map. Why?

Abstract (arXiv)

This paper considers a distributed reinforcement learning problem for decentralized linear quadratic control with partial state observations and local costs. We propose a Zero-Order Distributed Policy Optimization algorithm (ZODPO) that learns linear local controllers in a distributed fashion, leveraging the ideas of policy gradient, zero-order optimization and consensus algorithms. In ZODPO, each agent estimates the global cost by consensus, and then conducts local policy gradient in parallel based on zero-order gradient estimation. ZODPO only requires limited communication and storage even in large-scale systems. Further, we investigate the nonasymptotic performance of ZODPO and show that the sample complexity to approach a stationary point is polynomial with the error tolerance's inverse and the problem dimensions, demonstrating the scalability of ZODPO. We also show that the controllers generated throughout ZODPO are stabilizing controllers with high probability. Lastly, we numerically test ZODPO on multi-zone HVAC systems.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 6; 2023: 9; 2024: 9; 2025: 3.

33 papers.

Sort:

  1. 2021Gradient play in stochastic games: stationary points, convergence, and sample complexitycited by 32
  2. 2021Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approachcited by 27
  3. 2021Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexitycited by 15
  4. 2024Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimizationcited by 12
  5. 2023Learning the Kalman Filter with Fine-Grained Sample Complexitycited by 9
  6. 2023Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradientcited by 9
  7. 2021On the Sample Complexity of Decentralized Linear Quadratic Regulator with Partially Nested Information Structurecited by 7
  8. 2022Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functionscited by 6
  9. 2022Minibatch Stochastic Three Points Method for Unconstrained Smooth Minimizationcited by 6
  10. 2022Towards a Theoretical Foundation of Policy Optimization for Learning Control Policiescited by 6
  11. 2022Global Convergence of Direct Policy Search for State-Feedback H_∞ Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferentialcited by 6
  12. 2023Learning the Uncertainty Sets for Control Dynamics via Set Membership: A Non-Asymptotic Analysiscited by 6
21 more
  1. 2021Asynchronous Distributed Reinforcement Learning for LQR Control via Zeroth-Order Block Coordinate Descentcited by 4
  2. 2023Global Convergence of Receding-Horizon Policy Search in Learning Estimator Designscited by 4
  3. 2023Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithmscited by 4
  4. 2023Learning Zero-Sum Linear Quadratic Games with Improved Sample Complexity and Last-Iterate Convergencecited by 3
  5. 2024Scalable spectral representations for multi-agent reinforcement learning in network MDPscited by 3
  6. 2021Scalable regret for learning to control network-coupled subsystems with unknown dynamicscited by 2
  7. 2024Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspectivecited by 2
  8. 2024Learning to Stabilize Unknown LTI Systems on a Single Trajectory under Stochastic Noisecited by 2
  9. 2024System stabilization with policy optimization on unstable latent manifoldscited by 2
  10. 2022Learning Decentralized Linear Quadratic Regulators with √T Regretcited by 1
  11. 2023Policy Evaluation in Decentralized POMDPs with Belief Sharingcited by 1
  12. 2023On the Hardness of Learning to Stabilize Linear Systemscited by 1
  13. 2024Model-Free Learning for the Linear Quadratic Regulator over Rate-Limited Channelscited by 1
  14. 2025Exploiting inter-agent coupling information for efficient reinforcement learning of cooperative LQRcited by 1
  15. 2022Secure Distributed/Federated Learning: Prediction-Privacy Trade-Off for Multi-Agent Systemcited by 0
  16. 2023Policy Evaluation in Distributional LQRcited by 0
  17. 2024Distributed Policy Gradient for Linear Quadratic Networked Control with Limited Communication Rangecited by 0
  18. 2024Two-Timescale Optimization Framework for Sparse-Feedback Linear-Quadratic Optimal Controlcited by 0
  19. 2024Heavy-Ball Momentum Accelerated Actor-Critic With Function Approximationcited by 0
  20. 2025Douglas-Rachford Splitting for Group-Sparse Feedback Linear-Quadratic Controlcited by 0
  21. 2025Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspectivecited by 0