An Output Feedback Q-learning Algorithm for Optimal Control of Nonlinear Systems with Koopman Linear Embedding
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
In the reinforcement learning literature, strong theoretical guarantees have been obtained for algorithms applicable to LTI systems. However, in the nonlinear case only weaker results have been obtained for algorithms that mostly rely on the use of function approximation strategies like, for example, neural networks. In this paper, we study the applicability of a known output-feedback Q-learning algorithm to the class of nonlinear systems that admit a Koopman linear embedding. This algorithm uses only input-output data, and no knowledge of either the system model or the Koopman lifting functions is required. Moreover, no function approximation techniques are used, and the same theoretical guarantees as for LTI systems are preserved. Furthermore, we analyze the performance of the algorithm when the Koopman linear embedding is only an approximation of the real nonlinear system. A simulation example verifies the applicability of this method.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 21 references that are in the map. First posted on arXiv 31 Mar 2026.
Builds on
8 of 21 references.
References · 8
- 2022Koopman Form of Nonlinear Systems with Inputscited by 11
- 2025An overview of Koopman-based control: From error bounds to closed-loop guaranteescited by 10
- 2021Efficient Off-Policy Q-Learning for Data-Based Discrete-Time LQR Problemscited by 9
- 2024Willems' Fundamental Lemma for Nonlinear Systems with Koopman Linear Embeddingcited by 5
- 2026On the Existence of Koopman Linear Embeddings for Controlled Nonlinear Systemscited by 3
- 2023Notes on data-driven output-feedback control of linear MIMO systemscited by 2
- 2025On the Exponential Stability of Koopman Model Predictive Controlcited by 2
- 2023An efficient data-based off-policy Q-learning algorithm for optimal output feedback control of linear systemscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Data-driven optimal control of unknown nonlinear dynamical systems using the Koopman operator
- 2024Learning Stable Koopman Embeddings for Identification and Control
- 2024Model-free optimal controller for discrete-time Markovian jump linear systems: A Q-learning approach
- 2026A Switching System Theory of Q-Learning with Linear Function Approximation
- 2024Koopman-Assisted Reinforcement Learning
- 2024Model Predictive Control of Nonlinear Dynamics Using Online Adaptive Koopman Operators ↗
- 2025Comparative Study of Q-Learning for State-Feedback LQG Control with an Unknown Model
- 2026Target Updates May Stabilize Linear Q-Learning: Periodic and Soft Dynamics