Tailoring Reproducing Kernels for Optimal Control via Policy Iteration
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
This paper presents a novel approach to formulating the actor-critic method for optimal control by casting policy iteration in reproducing kernel Hilbert spaces (RKHSs -- also known as native spaces). By tailoring the reproducing kernel and RKHS to the dynamics of the nonlinear optimal control problem, we leverage recent advancements in characterizing error bounds from statistical and machine learning theory. These approximations define a general strategy to select the bases of the actor-critic networks, and we formally guarantee for the first time that this basis selection procedure leads to closed-form error bounds for the individual steps of policy iteration. These bounds often have a geometric and computable form, making them potentially useful for a priori or a posteriori evaluation of candidate collections of scattered bases. Numerical studies subsequently provide qualitative evidence of the practical performance achieved for the full recursion using the algorithms and theory developed for the single-step error bounds.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 38 references that are in the map. First posted on arXiv 26 Oct 2025.
Builds on
4 of 38 references.
References · 4
- 2019Robust feedback control of nonlinear PDEs by numerical approximation of high-dimensional Hamilton-Jacobi-Isaacs equationscited by 6
- 2020Policy iteration for Hamilton-Jacobi-Bellman equations with control constraintscited by 2
- 2023Hermite kernel surrogates for the value function of high-dimensional nonlinear optimal control problemscited by 2
- 2024Convergence Rates of Online Critic Value Function Approximation in Native Spacescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Sampling Complexity of TD and PPO in RKHS
- 2025Policy Newton Algorithm in Reproducing Kernel Hilbert Space
- 2024Compatible Gradient Approximations for Actor-Critic Algorithms
- 2026Global Convergence of Average Reward Constrained MDPs with Neural Critic and General Policy Parameterization
- 2026Wasserstein Formulation of Reinforcement Learning. An Optimal Transport Perspective on Policy Optimization
- 2024Closing the Gap: Achieving Global Convergence (Last Iterate) of Actor-Critic under Markovian Sampling with Neural Network Parametrization
- 2025Safe exploration in reproducing kernel Hilbert spaces
- 2024Neural Network Approaches for Parameterized Optimal Control