mlsanityData to 14 Oct 2026

Interacting Particle Systems for Fast Linear Quadratic RL

arXiv 2406.11057 ↗First posted 16 Jun 2024eess.SY cs.SYcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This paper is concerned with the design of algorithms based on systems of interacting particles to represent, approximate, and learn the optimal control law for reinforcement learning (RL). The primary contribution is that convergence rates are greatly accelerated by the interactions between particles. Theory focuses on the linear quadratic stochastic optimal control problem for which a complete and novel theory is presented. Apart from the new algorithm, sample complexity bounds are obtained, and it is shown that the mean square error scales as 1/N where N is the number of particles. The theoretical results and algorithms are illustrated with numerical experiments and comparisons with other recent approaches, where the faster convergence of the proposed algorithm is numerically demonstrated.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 34 references that are in the map. First posted on arXiv 16 Jun 2024.

Builds on

14 of 34 references.

References · 14

  1. 2017Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Drivingcited by 90
  2. 2020Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizoncited by 35
  3. 2016Highly-Smooth Zero-th Order Online Optimization Vianney Perchetcited by 23
  4. 2019Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulatorcited by 23
  5. 2018Model-Free Linear Quadratic Control via Reduction to Expert Predictioncited by 14
  6. 2014Path Integral Control and State Dependent Feedbackcited by 12
  7. 2021Online Policy Gradient for Model Free Learning of Linear Quadratic Regulators with √T Regretcited by 12
  8. 2023A Survey of Feedback Particle Filter and related Controlled Interacting Particle Systems (CIPS)cited by 5
  9. 2019On the Convergence of the Iterative Linear Exponential Quadratic Gaussian Algorithm to Stationary Pointscited by 4
  10. 2020On the Mathematical Theory of Ensemble (Linear-Gaussian) Kalman-Bucy Filteringcited by 4
  11. 2023Learning Zero-Sum Linear Quadratic Games with Improved Sample Complexity and Last-Iterate Convergencecited by 4
  12. 2021Controlled Interacting Particle Algorithms for Simulation-based Reinforcement Learningcited by 3
2 more
  1. 2016On the Stability of Kalman-Bucy Diffusion Processescited by 2
  2. 2018On the stability of matrix-valued Riccati diffusionscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Optimistic Training and Convergence of Q-Learning -- Extended Versioncited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problems
  2. 2026Mathematical methods of reinforcement learning
  3. 2024Scalable Reinforcement Learning for Linear-Quadratic Control of Networks
  4. 2023Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization
  5. 2024Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noise
  6. 2026Reinforcement Learning Method for Zero-Sum Linear-Quadratic Stochastic Differential Games in Infinite Horizons
  7. 2026Heavy-Ball Q-Learning with Residual Weighting Correction
  8. 2025Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control