# Power-Constrained Policy Gradient Methods for LQR

arXiv: https://arxiv.org/abs/2507.15806 · HTML: https://mlsanity.com/paper/2507.15806/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 26 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 13 of 13)

- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication](https://mlsanity.com/paper/1902.00340/) (2019)
- [Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation](https://mlsanity.com/paper/2105.14368/) (2021)
- [Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems](https://mlsanity.com/paper/1812.08305/) (2018)
- [SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation](https://mlsanity.com/paper/2006.10311/) (2020)
- [Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraints](https://mlsanity.com/paper/2106.00490/) (2021)
- [Online Policy Gradient for Model Free Learning of Linear Quadratic Regulators with √T Regret](https://mlsanity.com/paper/2102.12608/) (2021)
- [Gradient Statistics Aware Power Control for Over-the-Air Federated Learning](https://mlsanity.com/paper/2003.02089/) (2020)
- [Fundamental limits of over-the-air optimization: Are analog schemes optimal?](https://mlsanity.com/paper/2109.05222/) (2021)
- [Model-Free Learning for the Linear Quadratic Regulator over Rate-Limited Channels](https://mlsanity.com/paper/2401.01258/) (2024)
- [Universal gradient descent](https://mlsanity.com/paper/1711.00394/) (2017)

## Abstract (arXiv)

Consider a discrete-time Linear Quadratic Regulator (LQR) problem solved using policy gradient descent when the system matrices are unknown. The gradient is transmitted across a noisy channel over a finite time horizon using analog communication by a transmitter with an average power constraint. This is a simple setup at the intersection of reinforcement learning and networked control systems. We first consider a communication-constrained optimization framework, where gradient descent is applied to optimize a non-convex function under noisy gradient transmission. We provide an optimal power allocation algorithm that minimizes an upper bound on the expected optimality error at the final iteration and show that adaptive power allocation can lead to better convergence rate as compared to standard gradient descent with uniform power distribution. We then apply our results to the LQR setting.