mlsanityData to 14 Oct 2026

Harnessing Data from Clustered LQR Systems: Personalized and Collaborative Policy Optimization

arXiv 2511.17489 ↗First posted 21 Nov 2025cs.LG cs.SY eess.SYcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

It is known that reinforcement learning (RL) is data-hungry. To improve sample-efficiency of RL, it has been proposed that the learning algorithm utilize data from 'approximately similar' processes. However, since the process models are unknown, identifying which other processes are similar poses a challenge. In this work, we study this problem in the context of the benchmark Linear Quadratic Regulator (LQR) setting. Specifically, we consider a setting with multiple agents, each corresponding to a copy of a linear process to be controlled. The agents' local processes can be partitioned into clusters based on similarities in dynamics and tasks. Combining ideas from sequential elimination and zeroth-order policy optimization, we propose a new algorithm that performs simultaneous clustering and learning to output a personalized policy (controller) for each cluster. Under a suitable notion of cluster separation that captures differences in closed-loop performance across systems, we prove that our approach guarantees correct clustering with high probability. Furthermore, we show that the sub-optimality gap of the policy learned for each cluster scales inversely with the size of the cluster, with no additional bias, unlike in prior works on collaborative learning-based control. Our work is the first to reveal how clustering can be used in data-driven control to learn personalized policies that enjoy statistical gains from collaboration but do not suffer sub-optimality due to inclusion of data from dissimilar processes. From a distributed implementation perspective, our method is attractive as it incurs only a mild logarithmic communication overhead.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 34 references that are in the map. First posted on arXiv 21 Nov 2025.

Builds on

25 of 34 references.

References · 25

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  3. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  4. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  6. 2019A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Normcited by 106
  7. 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 67
  8. 2019Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergencecited by 46
  9. 2022Multi-Task Imitation Learning for Linear Dynamical Systemscited by 23
  10. 2023Model-free Learning with Heterogeneous Dynamical Systems: A Federated LQR Approachcited by 23
  11. 2022FedSysID: A Federated Approach to Sample-Efficient System Identificationcited by 17
  12. 2024Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQRcited by 17
13 more
  1. 2021Joint Learning of Linear Time-Invariant Dynamical Systemscited by 16
  2. 2023Learning Personalized Models with Clustered System Identificationcited by 11
  3. 2023Multi-Task System Identification of Similar Linear Time-Invariant Dynamical Systemscited by 8
  4. 2023Learning Dynamical Systems by Leveraging Data from Similar Systemscited by 8
  5. 2023Imitation and Transfer Learning for LQG Controlcited by 8
  6. 2024Sample Complexity of the Linear Quadratic Regulator: A Reinforcement Learning Lenscited by 8
  7. 2022Global Convergence of Federated Learning for Mixed Regressioncited by 6
  8. 2025Domain Randomization is Sample Efficient for Linear Quadratic Controlcited by 5
  9. 2024Regret Analysis of Multi-task Representation Learning for Linear-Quadratic Adaptive Controlcited by 4
  10. 2024A Moreau Envelope Approach for LQR Meta-Policy Estimationcited by 2
  11. 2025Policy Gradient Bounds in Multitask LQRcited by 2
  12. 2025Learning clusters of partially observed linear dynamical systemscited by 1
  13. 2025Federated Nonlinear System Identificationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Scalar Federated Learning for Linear Quadratic Regulatorcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Policy-Based Trajectory Clustering in Offline Reinforcement Learning
  2. 2025Asymptotically optimal reinforcement learning in Block Markov Decision Processes
  3. 2026Scalable Policy Optimization for Networked Multi-Agent Reinforcement Learning with Continuous State-Action Spaces
  4. 2024ClusterComm: Discrete Communication in Decentralized MARL using Internal Representation Clustering
  5. 2019Reinforcement Learning with Policy Mixture Model for Temporal Point Processes Clustering ↗
  6. 2026Collaborative Yet Personalized Policy Training: Single-Timescale Federated Actor-Critic
  7. 2025Adaptive Policy Synchronization for Scalable Reinforcement Learning
  8. 2025Linear Dynamics meets Linear MDPs: Closed-Form Optimal Policies via Reinforcement Learning