# Learning to Reach, Swim, Walk and Fly in One Trial: Data-Driven Control with Scarce Data and Side Information

arXiv: https://arxiv.org/abs/2106.10533 · HTML: https://mlsanity.com/paper/2106.10533/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 53 references that are in the map.
- First posted on arXiv 19 Jun 2021.

## Builds on (most cited first, 25 of 25)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [Distributed Distributional Deterministic Policy Gradients](https://mlsanity.com/paper/1804.08617/) (2018)
- [Continuous Deep Q-Learning with Model-based Acceleration](https://mlsanity.com/paper/1603.00748/) (2016)
- [Acme: A Research Framework for Distributed Reinforcement Learning](https://mlsanity.com/paper/2006.00979/) (2020)
- [Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control](https://mlsanity.com/paper/1611.03537/) (2016)
- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [Control of Memory, Active Perception, and Action in Minecraft](https://mlsanity.com/paper/1605.09128/) (2016)
- [Data-Enabled Predictive Control: In the Shallows of the DeePC](https://mlsanity.com/paper/1811.05890/) (2018)
- [Data-Driven Model Predictive Control with Stability and Robustness Guarantees](https://mlsanity.com/paper/1906.04679/) (2019)
- [Data informativity: a new perspective on data-driven analysis and control](https://mlsanity.com/paper/1908.00468/) (2019)
- [Learning Dynamical Systems with Side Information](https://mlsanity.com/paper/2008.10135/) (2020)
- [Data-Driven Reachable Set Computation using Adaptive Gaussian Process Classification and Monte Carlo Methods](https://mlsanity.com/paper/1910.02500/) (2019)
- [L1-GP: L1 Adaptive Control with Bayesian Learning](https://mlsanity.com/paper/2004.14594/) (2020)
- [From noisy data to feedback controllers: non-conservative design via a matrix S-lemma](https://mlsanity.com/paper/2006.00870/) (2020)
- [Combining Prior Knowledge and Data for Robust Controller Design](https://mlsanity.com/paper/2009.05253/) (2020)
- [Successive Convexification: A Superlinearly Convergent Algorithm for Non-convex Optimal Control Problems](https://mlsanity.com/paper/1804.06539/) (2018)
- [On-The-Fly Control of Unknown Smooth Systems from Limited Data](https://mlsanity.com/paper/2009.12733/) (2020)
- [On-The-Fly Control of Unknown Systems: From Side Information to Performance Guarantees through Reachability](https://mlsanity.com/paper/2011.05524/) (2020)

## Built on by (most cited first, 1 of 1)

- [Physics-Informed Kernel Embeddings: Integrating Prior System Knowledge with Data-Driven Control](https://mlsanity.com/paper/2301.03565/) (2023)

## Abstract (arXiv)

We develop a learning-based control algorithm for unknown dynamical systems under very severe data limitations. Specifically, the algorithm has access to streaming and noisy data only from a single and ongoing trial. It accomplishes such performance by effectively leveraging various forms of side information on the dynamics to reduce the sample complexity. Such side information typically comes from elementary laws of physics and qualitative properties of the system. More precisely, the algorithm approximately solves an optimal control problem encoding the system's desired behavior. To this end, it constructs and iteratively refines a data-driven differential inclusion that contains the unknown vector field of the dynamics. The differential inclusion, used in an interval Taylor-based method, enables to over-approximate the set of states the system may reach. Theoretically, we establish a bound on the suboptimality of the approximate solution with respect to the optimal control with known dynamics. We show that the longer the trial or the more side information is available, the tighter the bound. Empirically, experiments in a high-fidelity F-16 aircraft simulator and MuJoCo's environments illustrate that, despite the scarcity of data, the algorithm can provide performance comparable to reinforcement learning algorithms trained over millions of environment interactions. Besides, we show that the algorithm outperforms existing techniques combining system identification and model predictive control.