mlsanityData to 7 Oct 2026

Information Theoretic Regret Bounds for Online Nonlinear Control

arXiv 2006.12466 ↗First posted 22 Jun 2020cs.LG cs.RO math.OCcited by 87

Not on the map. Why?

Abstract (arXiv)

This work studies the problem of sequential control in an unknown, nonlinear dynamical system, where we model the underlying system dynamics as an unknown function in a known Reproducing Kernel Hilbert Space. This framework yields a general setting that permits discrete and continuous control inputs as well as non-smooth, non-differentiable dynamics. Our main result, the Lower Confidence-based Continuous Control (LC3) algorithm, enjoys a near-optimal O(√T) regret bound against the optimal controller in episodic settings, where T is the number of episodes. The bound has no explicit dependence on dimension of the system dynamics, which could be infinite, but instead only depends on information theoretic quantities. We empirically show its application to a number of nonlinear control tasks and demonstrate the benefit of exploration for learning model dynamics.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 20; 2023: 15; 2024: 16; 2025: 6; 2026: 7.

87 papers.

Sort:

By area:

  1. 2021Bilinear Classes: A Structural Framework for Provable Generalization in RLcited by 176
  2. 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 160
  3. 2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 128
  4. 2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
  5. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  6. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  7. 2021Instrumental Variable Value Iteration for Causal Offline Reinforcement Learningcited by 35
  8. 2021Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvaturecited by 32
  9. 2023Optimistic Active Exploration of Dynamical Systemscited by 32
  10. 2022Model-based RL with Optimistic Posterior Sampling: Structural Conditions and Sample Complexitycited by 31
  11. 2022A General Framework for Sample-Efficient Function Approximation in Reinforcement Learningcited by 31
  12. 2021Robust Policy Gradient against Strong Data Corruptioncited by 30
38 more
  1. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  2. 2023The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learningcited by 27
  3. 2021Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coveragecited by 26
  4. 2022Single Trajectory Nonparametric Learning of Nonlinear Dynamicscited by 26
  5. 2022Proximal Point Imitation Learningcited by 25
  6. 2021Stabilizing Dynamical Systems via Policy Gradient Methodscited by 24
  7. 2021A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learningcited by 24
  8. 2021Meta-Adaptive Nonlinear Control: Theory and Algorithmscited by 23
  9. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 22
  10. 2023Efficient Exploration in Continuous-time Model-based Reinforcement Learningcited by 21
  11. 2021On Reward-Free RL with Kernel and Neural Function Approximations: Single-Agent MDP and Markov Gamecited by 20
  12. 2023Optimal Exploration for Model-Based RL in Nonlinear Systemscited by 17
  13. 2023Distributionally Robust Model-based Reinforcement Learning with Large State Spacescited by 17
  14. 2023Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEscited by 16
  15. 2021Task-Optimal Exploration in Linear Dynamical Systemscited by 15
  16. 2022Learning Mixtures of Linear Dynamical Systemscited by 15
  17. 2022Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiencycited by 15
  18. 2021PC-MLP: Model-based Reinforcement Learning with Policy Cover Guided Explorationcited by 14
  19. 2022When to Update Your Model: Constrained Model-based Reinforcement Learningcited by 14
  20. 2023Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregressioncited by 14
  21. 2022Online Control of Unknown Time-Varying Dynamical Systemscited by 13
  22. 2023The Virtues of Laziness in Model-based RL: A Unified Objective and Algorithmscited by 12
  23. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximationcited by 11
  24. 2022Non-Linear Reinforcement Learning in Large Action Spaces: Structural Conditions and Sample-efficiency of Posterior Samplingcited by 10
  25. 2022Learning to Control Linear Systems can be Hardcited by 10
  26. 2021Sparsity in Partially Controllable Linear Systemscited by 9
  27. 2024When to Sense and Control? A Time-adaptive Approach for Continuous-Time RLcited by 9
  28. 2023Sample Complexity of Kernel-Based Q-Learningcited by 8
  29. 2024Switching the Loss Reduces the Cost in Batch (Offline) Reinforcement Learningcited by 8
  30. 2024Tighter Confidence Bounds for Sequential Kernel Regressioncited by 7
  31. 2024Active Learning for Control-Oriented Identification of Nonlinear Systemscited by 7
  32. 2025SOMBRL: Scalable and Optimistic Model-Based RLcited by 7
  33. 2021Provable Regret Bounds for Deep Online Learning and Controlcited by 6
  34. 2023Nonasymptotic Regret Analysis of Adaptive Linear Quadratic Control with Model Misspecificationcited by 6
  35. 2024No-Regret Reinforcement Learning in Smooth MDPscited by 6
  36. 2026Safe Exploration via Policy Priorscited by 6
  37. 2021Koopman Spectrum Nonlinear Regulators and Efficient Online Learningcited by 5
  38. 2021Regret Analysis of Learning-Based MPC with Partially-Unknown Cost Functioncited by 5

50 most cited of 87 All 87 (JSON)