mlsanityData to 7 Oct 2026

Logarithmic Regret for Nonlinear Control

arXiv 2501.10261 ↗First posted 17 Jan 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We address the problem of learning to control an unknown nonlinear dynamical system through sequential interactions. Motivated by high-stakes applications in which mistakes can be catastrophic, such as robotics and healthcare, we study situations where it is possible for fast sequential learning to occur. Fast sequential learning is characterized by the ability of the learning agent to incur logarithmic regret relative to a fully-informed baseline. We demonstrate that fast sequential learning is achievable in a diverse class of continuous control problems where the system dynamics depend smoothly on unknown parameters, provided the optimal control policy is persistently exciting. Additionally, we derive a regret bound which grows with the square root of the number of interactions for cases where the optimal policy is not persistently exciting. Our results provide the first regret bounds for controlling nonlinear dynamical systems depending nonlinearly on unknown parameters. We validate the trends our theory predicts in simulation on a simple dynamical system.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 17 Jan 2025.

Builds on

29 of 44 references.

References · 29

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  3. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 211
  4. 2020Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processescited by 199
  5. 2018Learning Without Mixing: Towards A Sharp Analysis of Linear System Identificationcited by 158
  6. 2017On the Sample Complexity of the Linear Quadratic Regulatorcited by 151
  7. 2020Naive Exploration is Optimal for Online LQRcited by 111
  8. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 105
  9. 2020Information Theoretic Regret Bounds for Online Nonlinear Controlcited by 87
  10. 2016Explore First, Exploit Next: The True Shape of Regret in Bandit Problemscited by 78
  11. 2020Active Learning for Nonlinear System Identification with Guaranteescited by 78
  12. 2017Finite Time Identification in Unstable Linear Systemscited by 66
17 more
  1. 2019Data Efficient Reinforcement Learning for Legged Robotscited by 60
  2. 2021Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrapcited by 48
  3. 2020Logarithmic Regret Bound in Partially Observable Linear Dynamical Systemscited by 46
  4. 2020Learning nonlinear dynamical systems from a single trajectorycited by 46
  5. 2020Non-asymptotic and Accurate Learning of Nonlinear Dynamical Systemscited by 39
  6. 2018Exploration in Structured Reinforcement Learningcited by 34
  7. 2020Logarithmic Regret for Learning Linear Quadratic Regulators Efficientlycited by 33
  8. 2022Learning with little mixingcited by 33
  9. 2023Optimistic Active Exploration of Dynamical Systemscited by 32
  10. 2020Regret Bounds for Adaptive Nonlinear Controlcited by 25
  11. 2023Optimal Exploration for Model-Based RL in Nonlinear Systemscited by 17
  12. 2021Minimal Expected Regret in Linear Quadratic Controlcited by 13
  13. 2022Regret Lower Bounds for Learning Linear Quadratic Gaussian Systemscited by 11
  14. 2024Sharp Rates in Dependent Learning Theory: Avoiding Sample Size Deflation for the Square Losscited by 10
  15. 2024Active Learning for Control-Oriented Identification of Nonlinear Systemscited by 7
  16. 2024Identification of Analytic Nonlinear Dynamical Systems with Non-asymptotic Guaranteescited by 7
  17. 2023Nonasymptotic Regret Analysis of Adaptive Linear Quadratic Control with Model Misspecificationcited by 6

Built on by

Not cited yet.