# Interacting Particle Systems for Fast Linear Quadratic RL

arXiv: https://arxiv.org/abs/2406.11057 · HTML: https://mlsanity.com/paper/2406.11057/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 14 of 14)

- [Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving](https://mlsanity.com/paper/1707.02342/) (2017)
- [Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon](https://mlsanity.com/paper/2006.15316/) (2020)
- [Highly-Smooth Zero-th Order Online Optimization Vianney Perchet](https://mlsanity.com/paper/1605.08165/) (2016)
- [Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulator](https://mlsanity.com/paper/1905.12842/) (2019)
- [Model-Free Linear Quadratic Control via Reduction to Expert Prediction](https://mlsanity.com/paper/1804.06021/) (2018)
- [Path Integral Control and State Dependent Feedback](https://mlsanity.com/paper/1406.4026/) (2014)
- [Online Policy Gradient for Model Free Learning of Linear Quadratic Regulators with √T Regret](https://mlsanity.com/paper/2102.12608/) (2021)
- [A Survey of Feedback Particle Filter and related Controlled Interacting Particle Systems (CIPS)](https://mlsanity.com/paper/2301.00935/) (2023)
- [On the Convergence of the Iterative Linear Exponential Quadratic Gaussian Algorithm to Stationary Points](https://mlsanity.com/paper/1910.08221/) (2019)
- [On the Mathematical Theory of Ensemble (Linear-Gaussian) Kalman-Bucy Filtering](https://mlsanity.com/paper/2006.08843/) (2020)
- [Learning Zero-Sum Linear Quadratic Games with Improved Sample Complexity and Last-Iterate Convergence](https://mlsanity.com/paper/2309.04272/) (2023)
- [Controlled Interacting Particle Algorithms for Simulation-based Reinforcement Learning](https://mlsanity.com/paper/2107.01244/) (2021)
- [On the Stability of Kalman-Bucy Diffusion Processes](https://mlsanity.com/paper/1610.04686/) (2016)
- [On the stability of matrix-valued Riccati diffusions](https://mlsanity.com/paper/1808.00235/) (2018)

## Built on by (most cited first, 1 of 1)

- [Optimistic Training and Convergence of Q-Learning -- Extended Version](https://mlsanity.com/paper/2602.06146/) (2026)

## Abstract (arXiv)

This paper is concerned with the design of algorithms based on systems of interacting particles to represent, approximate, and learn the optimal control law for reinforcement learning (RL). The primary contribution is that convergence rates are greatly accelerated by the interactions between particles. Theory focuses on the linear quadratic stochastic optimal control problem for which a complete and novel theory is presented. Apart from the new algorithm, sample complexity bounds are obtained, and it is shown that the mean square error scales as 1/N where N is the number of particles. The theoretical results and algorithms are illustrated with numerical experiments and comparisons with other recent approaches, where the faster convergence of the proposed algorithm is numerically demonstrated.