# On the System Theoretic Offline Learning of Continuous-Time LQR with Exogenous Disturbances

arXiv: https://arxiv.org/abs/2509.16746 · HTML: https://mlsanity.com/paper/2509.16746/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Simple random search provides a competitive approach to reinforcement learning](https://mlsanity.com/paper/1803.07055/) (2018)
- [Formulas for Data-driven Control: Stabilization, Optimality and Robustness](https://mlsanity.com/paper/1903.06842/) (2019)
- [Reduced-Dimensional Reinforcement Learning Control using Singular Perturbation Approximations](https://mlsanity.com/paper/2004.14501/) (2020)
- [Reinforcement Learning of Structured Control for Linear Systems with Unknown State Matrix](https://mlsanity.com/paper/2011.01128/) (2020)
- [Analyzing the Impact of Computation in Adaptive Dynamic Programming for Stochastic LQR Problem](https://mlsanity.com/paper/2402.09575/) (2024)

## Abstract (arXiv)

We analyze offline designs of linear quadratic regulator (LQR) strategies with uncertain disturbances. First, we consider the scenario where the exogenous variable can be estimated in a controlled environment, and subsequently, consider a more practical and challenging scenario where it is unknown in a stochastic setting. Our approach builds on the fundamental learning-based framework of adaptive dynamic programming (ADP), combined with a Lyapunov-based analytical methodology to design the algorithms and derive sample-based approximations motivated from the Markov decision process (MDP)-based approaches. For the scenario involving non-measurable disturbances, we further establish stability and convergence guarantees for the learned control gains under sample-based approximations. The overall methodology emphasizes simplicity while providing rigorous guarantees. Finally, numerical experiments focus on the intricacies and validations for the design of offline continuous-time LQR with exogenous disturbances.