Thompson Sampling-Based Learning and Control for Unknown Dynamic Systems
Not on the map. Why?
Abstract (arXiv)
Thompson sampling (TS) is a Bayesian randomized exploration strategy that samples options (e.g., system parameters or control laws) from the current posterior and then applies the selected option that is optimal for a task, thereby balancing exploration and exploitation; this makes TS effective for active learning-based controller design. However, TS relies on finite parametric representations, which limits its applicability to more general spaces, which are more commonly encountered in control system design. To address this issue, this work proposes a parameterization method for control law learning using reproducing kernel Hilbert spaces and designs a data-driven active learning control approach. Specifically, the proposed method treats the control law as an element in a function space, allowing the design of control laws without imposing restrictions on the system structure or the form of the controller. A TS framework is proposed in this work to reduce control costs through online exploration and exploitation, and the convergence guarantees are further provided for the learning process. Theoretical analysis shows that the proposed method learns the relationship between control laws and closed-loop performance metrics at an exponential rate, and the upper bound of control regret is also derived. Furthermore, the closed-loop stability of the proposed learning framework is analyzed. Numerical experiments on controlling unknown nonlinear systems validate the effectiveness of the proposed method.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 53 references that are in the map. First posted on arXiv 27 Jun 2025.
Builds on
24 of 53 references.
References · 24
- 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
- 2017A Tutorial on Thompson Samplingcited by 433
- 2014Predictive Entropy Search for Efficient Global Optimization of Black-box Functionscited by 204
- 2021Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and controlcited by 137
- 2020Fast active learning for pure exploration in reinforcement learningcited by 92
- 2019Formulas for Data-driven Control: Stabilization, Optimality and Robustnesscited by 47
- 2020Scalable Thompson Sampling using Sparse Gaussian Process Modelscited by 39
- 2017Taming Non-stationary Bandits: A Bayesian Approachcited by 32
- 2019Active Learning of Dynamics for Data-Driven Control Using Koopman Operatorscited by 32
- 2017Learning Algorithms for Active Learningcited by 30
- 2015Upper-Confidence-Bound Algorithms for Active Learning in Multi-Armed Banditscited by 29
- 2022On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RLcited by 27
12 more
- 2021Active Learning in Robotics: A Review of Control Principlescited by 26
- 2019Solving Bernoulli Rank-One Bandits with Unimodal Thompson Samplingcited by 23
- 2024Sliding-Window Thompson Sampling for Non-Stationary Settingscited by 17
- 2017Thompson Sampling for Linear-Quadratic Control Problemscited by 14
- 2022Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Controlcited by 13
- 2023Data-driven Methods Applied to Soft Robot Modeling and Control: A Reviewcited by 13
- 2020Localized active learning of Gaussian process state space modelscited by 8
- 2022Thompson Sampling Achieves tilde O(√T) Regret in Linear Quadratic Controlcited by 8
- 2019Robust exploration in linear quadratic reinforcement learningcited by 2
- 2021Direct data-driven control of LTV systemscited by 2
- 2023Sequential learning and control: Targeted exploration for robust performancecited by 1
- 2023Dual Control of Exploration and Exploitation for Auto-Optimisation Control with Active Learningcited by 1
Built on by
Not cited yet.