mlsanityData to 7 Oct 2026

Learning the Kalman Filter with Fine-Grained Sample Complexity

arXiv 2301.12624 ↗First posted 30 Jan 2023math.OC cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

We develop the first end-to-end sample complexity of model-free policy gradient (PG) methods in discrete-time infinite-horizon Kalman filtering. Specifically, we introduce the receding-horizon policy gradient (RHPG-KF) framework and demonstrate tildeO(ε-2) sample complexity for RHPG-KF in learning a stabilizing filter that is ε-close to the optimal Kalman filter. Notably, the proposed RHPG-KF framework does not require the system to be open-loop stable nor assume any prior knowledge of a stabilizing filter. Our results shed light on applying model-free PG methods to control a linear dynamical system where the state measurements could be corrupted by statistical noises and other (possibly adversarial) disturbances.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 17 of its 37 references that are in the map. First posted on arXiv 30 Jan 2023.

Builds on

17 of 37 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 211
  6. 2013Optimal rates for zero-order convex optimization: the power of two function evaluationscited by 190
  7. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 129
  8. 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 56
  9. 2020Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizoncited by 43
  10. 2019Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problemcited by 35
  11. 2019Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approachcited by 33
  12. 2021Stabilizing Dynamical Systems via Policy Gradient Methodscited by 24
5 more
  1. 2021Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Controlcited by 23
  2. 2019Learning the Globally Optimal Distributed LQ Regulatorcited by 19
  3. 2021Derivative-Free Policy Optimization for Linear Risk-Sensitive and Robust Control Design: Implicit Regularization and Sample Complexitycited by 15
  4. 2022Globally Convergent Policy Search over Dynamic Filters for Output Estimationcited by 8
  5. 2022Connectivity of the Feasible and Sublevel Sets of Dynamic Output Feedback Control with Robustness Constraintscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 1; 2026: 1.

9 papers.

  1. 2023Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradientcited by 9
  2. 2024Decision Transformer as a Foundation Model for Partially Observable Continuous Controlcited by 5
  3. 2023Global Convergence of Receding-Horizon Policy Search in Learning Estimator Designscited by 4
  4. 2023Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithmscited by 4
  5. 2024Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspectivecited by 2
  6. 2024Policy Optimization for PDE Control with a Warm Startcited by 1
  7. 2024Structure Matters: Dynamic Policy Gradientcited by 1
  8. 2025Logarithmic Regret and Polynomial Scaling in Online Multi-step-ahead Predictioncited by 1
  9. 2026A Memory Efficient Unified Algorithm for Online Learning of Linear Dynamical Systemscited by 0