# Robustness of Online Identification-based Policy Iteration to Noisy Data

arXiv: https://arxiv.org/abs/2504.07627 · HTML: https://mlsanity.com/paper/2504.07627/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2025.

## Built on by (most cited first, 3 of 3)

- [Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches](https://mlsanity.com/paper/2505.03706/) (2025)
- [Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression](https://mlsanity.com/paper/2512.03764/) (2025)
- [A Stochastic Gradient Descent Approach to Design Policy Gradient Methods for LQR](https://mlsanity.com/paper/2602.18933/) (2026)

## Abstract (arXiv)

This article investigates the core mechanisms of indirect data-driven control for unknown systems, focusing on the application of policy iteration (PI) within the context of the linear quadratic regulator (LQR) optimal control problem. Specifically, we consider a setting where data is collected sequentially from a linear system subject to exogenous process noise, and is then used to refine estimates of the optimal control policy. We integrate recursive least squares (RLS) for online model estimation within a certainty-equivalent framework, and employ PI to iteratively update the control policy. In this work, we investigate first the convergence behavior of RLS under two different models of adversarial noise, namely point-wise and energy bounded noise, and then we provide a closed-loop analysis of the combined model identification and control design process. This iterative scheme is formulated as an algorithmic dynamical system consisting of the feedback interconnection between two algorithms expressed as discrete-time systems. This system theoretic viewpoint on indirect data-driven control allows us to establish convergence guarantees to the optimal controller in the face of uncertainty caused by noisy data. Simulations illustrate the theoretical results.