# Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

arXiv: https://arxiv.org/abs/2107.07788 · HTML: https://mlsanity.com/paper/2107.07788/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 59 references that are in the map.
- First posted on arXiv 16 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon](https://mlsanity.com/paper/2006.15316/) (2020)
- [Robust Policy Iteration for Continuous-time Linear Quadratic Regulation](https://mlsanity.com/paper/2005.09528/) (2020)
- [Multiagent Rollout and Policy Iteration for POMDP with Application to Multi-Robot Repair Problems](https://mlsanity.com/paper/2011.04222/) (2020)
- [From Reinforcement Learning to Optimal Control: A unified framework for sequential decisions](https://mlsanity.com/paper/1912.03513/) (2019)
- [Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation](https://mlsanity.com/paper/2008.11592/) (2020)
- [Continuous-Time Robust Dynamic Programming](https://mlsanity.com/paper/1809.05867/) (2018)
- [A Performance and Stability Analysis of Low-inertia Power Grids with Stochastic System Inertia](https://mlsanity.com/paper/1903.00635/) (2019)
- [Approximate Dynamic Programming For Linear Systems with State and Input Constraints](https://mlsanity.com/paper/1906.11369/) (2019)

## Built on by (most cited first, 8 of 8)

- [Inverse reinforcement learning by expert imitation for the stochastic linear-quadratic optimal control problem](https://mlsanity.com/paper/2405.17085/) (2024)
- [Robust policy iteration for continuous-time stochastic H_∞ control problem with unknown dynamics](https://mlsanity.com/paper/2402.04721/) (2024)
- [Model-free H∞ control of Ito stochastic system via off-policy reinforcement learning](https://mlsanity.com/paper/2403.04412/) (2024)
- [A Fully Data-Driven Value Iteration for Stochastic LQR: Convergence, Robustness and Stability](https://mlsanity.com/paper/2505.02970/) (2025)
- [Learning-based primal-dual optimal control of discrete-time stochastic systems with multiplicative noise](https://mlsanity.com/paper/2506.02613/) (2025)
- [Comparative Study of Q-Learning for State-Feedback LQG Control with an Unknown Model](https://mlsanity.com/paper/2511.07870/) (2025)
- [Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming](https://mlsanity.com/paper/2512.20911/) (2025)
- [Reinforcement Learning Method for Zero-Sum Linear-Quadratic Stochastic Differential Games in Infinite Horizons](https://mlsanity.com/paper/2602.08075/) (2026)

## Abstract (arXiv)

This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given by the proposed optimistic least-squares-based policy iteration are proved to converge to a small neighborhood of the optimal solution with probability one, under mild conditions. The application of the proposed algorithm to a triple inverted pendulum example validates its feasibility and effectiveness.