# On-the-fly control of unknown nonlinear systems with sublinear regret

arXiv: https://arxiv.org/abs/2206.11103 · HTML: https://mlsanity.com/paper/2206.11103/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2022.

## Built on by (most cited first, 1 of 1)

- [Probabilities Are Not Enough: Formal Controller Synthesis for Stochastic Dynamical Models with Epistemic Uncertainty](https://mlsanity.com/paper/2210.05989/) (2022)

## Abstract (arXiv)

We study the problem of data-driven, constrained control of unknown nonlinear dynamics from a single ongoing and finite-horizon trajectory. We consider a one-step optimal control problem with a smooth, black-box objective, typically a composition of a known cost function and the unknown dynamics. We investigate an on-the-fly control paradigm, i.e., at each time step, the evolution of the dynamics and the first-order information of the cost are provided only for the executed control action. We propose an optimization-based control algorithm that iteratively minimizes a data-driven surrogate function for the unknown objective. We prove that the proposed approach incurs sublinear cumulative regret (step-wise suboptimality with respect to an optimal one-step controller) and is worst-case optimal among a broad class of data-driven control algorithms. We also present tractable reformulations of the approach that can leverage off-the-shelf solvers for efficient implementations.