# Nonparametric adaptive control and prediction: theory and randomized algorithms

arXiv: https://arxiv.org/abs/2106.03589 · HTML: https://mlsanity.com/paper/2106.03589/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 51 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [On Exact Computation with an Infinitely Wide Neural Net](https://mlsanity.com/paper/1904.11955/) (2019)
- [A Mean Field View of the Landscape of Two-Layers Neural Networks](https://mlsanity.com/paper/1804.06561/) (2018)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Linearized two-layers neural networks in high dimension](https://mlsanity.com/paper/1904.12191/) (2019)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [When Do Neural Networks Outperform Kernel Methods?](https://mlsanity.com/paper/2006.13409/) (2020)
- [A vector-contraction inequality for Rademacher complexities](https://mlsanity.com/paper/1605.00251/) (2016)
- [On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions](https://mlsanity.com/paper/1502.06800/) (2015)
- [Trainability and Accuracy of Neural Networks: An Interacting Particle System Approach](https://mlsanity.com/paper/1805.00915/) (2018)
- [Learning Stability Certificates from Data](https://mlsanity.com/paper/2008.05952/) (2020)
- [Exponentiated Gradient Meets Gradient Descent](https://mlsanity.com/paper/1902.01903/) (2019)
- [Learning Stabilizable Nonlinear Dynamics with Contraction-Based Regularization](https://mlsanity.com/paper/1907.13122/) (2019)
- [Regret Bounds for Adaptive Nonlinear Control](https://mlsanity.com/paper/2011.13101/) (2020)
- [Random Fourier Features for Operator-Valued Kernels](https://mlsanity.com/paper/1605.02536/) (2016)
- [Learning Contracting Vector Fields For Stable Imitation Learning](https://mlsanity.com/paper/1804.04878/) (2018)
- [Operator-Valued Bochner Theorem, Fourier Feature Maps for Operator-Valued Kernels, and Vector-Valued Learning](https://mlsanity.com/paper/1608.05639/) (2016)
- [Implicit Regularization and Momentum Algorithms in Nonlinearly Parameterized Adaptive Control and Prediction](https://mlsanity.com/paper/1912.13154/) (2019)
- [Adaptive Estimation for Nonlinear Systems using Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/1707.01567/) (2017)

## Built on by (most cited first, 8 of 8)

- [Chefs' Random Tables: Non-Trigonometric Random Features](https://mlsanity.com/paper/2205.15317/) (2022)
- [Control-oriented meta-learning](https://mlsanity.com/paper/2204.06716/) (2022)
- [FAVOR#: Sharp Attention Kernel Approximations via New Classes of Positive Random Features](https://mlsanity.com/paper/2302.00787/) (2023)
- [Learning of Hamiltonian Dynamics with Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2312.09734/) (2023)
- [Learning Hamiltonian Dynamics with Reproducing Kernel Hilbert Spaces and Random Features](https://mlsanity.com/paper/2404.07703/) (2024)
- [Learning dissipative Hamiltonian dynamics with reproducing kernel Hilbert spaces and random Fourier features](https://mlsanity.com/paper/2410.18656/) (2024)
- [Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem](https://mlsanity.com/paper/2411.15972/) (2024)
- [Self-Adaptive Learning and Model Predictive Control for Tracking Unknown Dynamics with No Regret](https://mlsanity.com/paper/2607.26370/) (2026)

## Abstract (arXiv)

A key assumption in the theory of nonlinear adaptive control is that the uncertainty of the system can be expressed in the linear span of a set of known basis functions. While this assumption leads to efficient algorithms, it limits applications to very specific classes of systems. We introduce a novel nonparametric adaptive algorithm that estimates an infinite-dimensional density over parameters online to learn an unknown dynamics in a reproducing kernel Hilbert space. Surprisingly, the resulting control input admits an analytical expression that enables its implementation despite its underlying infinite-dimensional structure. While this adaptive input is rich and expressive - subsuming, for example, traditional linear parameterizations - its computational complexity grows linearly with time, making it comparatively more expensive than its parametric counterparts. Leveraging the theory of random Fourier features, we provide an efficient randomized implementation that recovers the complexity of classical parametric methods while provably retaining the expressivity of the nonparametric input. In particular, our explicit bounds only depend polynomially on the underlying parameters of the system, allowing our proposed algorithms to efficiently scale to high-dimensional systems. As an illustration of the method, we demonstrate the ability of the randomized approximation algorithm to learn a predictive model of a 60-dimensional system consisting of ten point masses interacting through Newtonian gravitation. By reinterpretation as a gradient flow on a specific loss, we conclude with a natural extension of our kernel-based adaptive algorithms to deep neural networks. We show empirically that the extra expressivity afforded by deep representations can lead to improved performance at the expense of closed-loop stability that is rigorously guaranteed and consistently observed for kernel machines.