# Structured Policy Iteration for Linear Quadratic Regulator

arXiv: https://arxiv.org/abs/2007.06202 · HTML: https://mlsanity.com/paper/2007.06202/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2020.

## Built on by (most cited first, 8 of 8)

- [Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models](https://mlsanity.com/paper/2404.08080/) (2024)
- [Learning Quantile Functions without Quantile Crossing for Distribution-free Time Series Forecasting](https://mlsanity.com/paper/2111.06581/) (2021)
- [Multivariate Quantile Function Forecaster](https://mlsanity.com/paper/2202.11316/) (2022)
- [Robust Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2202.11910/) (2022)
- [Robust Multivariate Time-Series Forecasting: Adversarial Attacks and Defense Mechanisms](https://mlsanity.com/paper/2207.09572/) (2022)
- [Adaptive Sampling for Probabilistic Forecasting under Distribution Shift](https://mlsanity.com/paper/2302.11870/) (2023)
- [Exploiting inter-agent coupling information for efficient reinforcement learning of cooperative LQR](https://mlsanity.com/paper/2504.20927/) (2025)
- [Douglas-Rachford Splitting for Group-Sparse Feedback Linear-Quadratic Control](https://mlsanity.com/paper/2507.19895/) (2025)

## Abstract (arXiv)

Linear quadratic regulator (LQR) is one of the most popular frameworks to tackle continuous Markov decision process tasks. With its fundamental theory and tractable optimal policy, LQR has been revisited and analyzed in recent years, in terms of reinforcement learning scenarios such as the model-free or model-based setting. In this paper, we introduce the textitStructured Policy Iteration (S-PI) for LQR, a method capable of deriving a structured linear policy. Such a structured policy with (block) sparsity or low-rank can have significant advantages over the standard LQR policy: more interpretable, memory-efficient, and well-suited for the distributed setting. In order to derive such a policy, we first cast a regularized LQR problem when the model is known. Then, our Structured Policy Iteration (S-PI) algorithm, which takes a policy evaluation step and a policy improvement step in an iterative manner, can solve this regularized LQR efficiently. We further extend the S-PI algorithm to the model-free setting where a smoothing procedure is adopted to estimate the gradient. In both the known-model and model-free setting, we prove convergence analysis under the proper choice of parameters. Finally, the experiments demonstrate the advantages of S-PI in terms of balancing the LQR performance and level of structure by varying the weight parameter.