# Harnessing Data from Clustered LQR Systems: Personalized and Collaborative Policy Optimization

arXiv: https://arxiv.org/abs/2511.17489 · HTML: https://mlsanity.com/paper/2511.17489/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 21 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 25 of 25)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm](https://mlsanity.com/paper/1902.03736/) (2019)
- [Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems](https://mlsanity.com/paper/1812.08305/) (2018)
- [Policy Optimization for H2 Linear Control with H_∞ Robustness Guarantee: Implicit Regularization and Global Convergence](https://mlsanity.com/paper/1910.09496/) (2019)
- [Multi-Task Imitation Learning for Linear Dynamical Systems](https://mlsanity.com/paper/2212.00186/) (2022)
- [Model-free Learning with Heterogeneous Dynamical Systems: A Federated LQR Approach](https://mlsanity.com/paper/2308.11743/) (2023)
- [FedSysID: A Federated Approach to Sample-Efficient System Identification](https://mlsanity.com/paper/2211.14393/) (2022)
- [Meta-Learning Linear Quadratic Regulators: A Policy Gradient MAML Approach for Model-free LQR](https://mlsanity.com/paper/2401.14534/) (2024)
- [Joint Learning of Linear Time-Invariant Dynamical Systems](https://mlsanity.com/paper/2112.10955/) (2021)
- [Learning Personalized Models with Clustered System Identification](https://mlsanity.com/paper/2304.01395/) (2023)
- [Multi-Task System Identification of Similar Linear Time-Invariant Dynamical Systems](https://mlsanity.com/paper/2301.01430/) (2023)
- [Learning Dynamical Systems by Leveraging Data from Similar Systems](https://mlsanity.com/paper/2302.04344/) (2023)
- [Imitation and Transfer Learning for LQG Control](https://mlsanity.com/paper/2303.09002/) (2023)
- [Sample Complexity of the Linear Quadratic Regulator: A Reinforcement Learning Lens](https://mlsanity.com/paper/2404.10851/) (2024)
- [Global Convergence of Federated Learning for Mixed Regression](https://mlsanity.com/paper/2206.07279/) (2022)
- [Domain Randomization is Sample Efficient for Linear Quadratic Control](https://mlsanity.com/paper/2502.12310/) (2025)
- [Regret Analysis of Multi-task Representation Learning for Linear-Quadratic Adaptive Control](https://mlsanity.com/paper/2407.05781/) (2024)
- [A Moreau Envelope Approach for LQR Meta-Policy Estimation](https://mlsanity.com/paper/2403.17364/) (2024)
- [Policy Gradient Bounds in Multitask LQR](https://mlsanity.com/paper/2509.19266/) (2025)
- [Learning clusters of partially observed linear dynamical systems](https://mlsanity.com/paper/2507.17638/) (2025)
- [Federated Nonlinear System Identification](https://mlsanity.com/paper/2508.15025/) (2025)

## Built on by (most cited first, 1 of 1)

- [Scalar Federated Learning for Linear Quadratic Regulator](https://mlsanity.com/paper/2604.05088/) (2026)

## Abstract (arXiv)

It is known that reinforcement learning (RL) is data-hungry. To improve sample-efficiency of RL, it has been proposed that the learning algorithm utilize data from 'approximately similar' processes. However, since the process models are unknown, identifying which other processes are similar poses a challenge. In this work, we study this problem in the context of the benchmark Linear Quadratic Regulator (LQR) setting. Specifically, we consider a setting with multiple agents, each corresponding to a copy of a linear process to be controlled. The agents' local processes can be partitioned into clusters based on similarities in dynamics and tasks. Combining ideas from sequential elimination and zeroth-order policy optimization, we propose a new algorithm that performs simultaneous clustering and learning to output a personalized policy (controller) for each cluster. Under a suitable notion of cluster separation that captures differences in closed-loop performance across systems, we prove that our approach guarantees correct clustering with high probability. Furthermore, we show that the sub-optimality gap of the policy learned for each cluster scales inversely with the size of the cluster, with no additional bias, unlike in prior works on collaborative learning-based control. Our work is the first to reveal how clustering can be used in data-driven control to learn personalized policies that enjoy statistical gains from collaboration but do not suffer sub-optimality due to inclusion of data from dissimilar processes. From a distributed implementation perspective, our method is attractive as it incurs only a mild logarithmic communication overhead.