# Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Ito Diffusions

arXiv: https://arxiv.org/abs/2105.10102 · HTML: https://mlsanity.com/paper/2105.10102/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 72 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 14 of 14)

- [Sorting out Lipschitz function approximation](https://mlsanity.com/paper/1811.05381/) (2018)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Error estimates for spectral convergence of the graph Laplacian on random geometric graphs towards the Laplace--Beltrami operator](https://mlsanity.com/paper/1801.10108/) (2018)
- [Approximation Bounds for Random Neural Networks and Reservoir Systems](https://mlsanity.com/paper/2002.05933/) (2020)
- [Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems](https://mlsanity.com/paper/1712.00754/) (2017)
- [Machine Learning for Prediction with Missing Dynamics](https://mlsanity.com/paper/1910.05861/) (2019)
- [Risk bounds for reservoir computing](https://mlsanity.com/paper/1910.13886/) (2019)
- [Model Reduction with Memory and the Machine Learning of Dynamical Systems](https://mlsanity.com/paper/1808.04258/) (2018)
- [Data-driven Discovery of Closure Models](https://mlsanity.com/paper/1803.09318/) (2018)
- [Perturbation theory for Markov chains via Wasserstein distance](https://mlsanity.com/paper/1503.04123/) (2015)
- [Non-parametric Estimation of Stochastic Differential Equations with Sparse Gaussian Processes](https://mlsanity.com/paper/1704.04375/) (2017)
- [Kernel-based Prediction of Non-Markovian Time Series](https://mlsanity.com/paper/2007.04286/) (2020)
- [Quantifying the accuracy of approximate diffusions and Markov chains](https://mlsanity.com/paper/1605.06420/) (2016)
- [Modeling of Missing Dynamical Systems: Deriving Parametric Models using a Nonparametric Framework](https://mlsanity.com/paper/1905.08082/) (2019)

## Built on by (most cited first, 5 of 5)

- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [Stationary Density Estimation of Ito Diffusions Using Deep Learning](https://mlsanity.com/paper/2109.03992/) (2021)
- [A non-intrusive machine learning framework for debiasing long-time coarse resolution climate simulations and quantifying rare events statistics](https://mlsanity.com/paper/2402.18484/) (2024)
- [A probabilistic framework for learning non-intrusive corrections to long-time climate simulations from short-time training data](https://mlsanity.com/paper/2408.02688/) (2024)
- [Goal-oriented learning of stochastic differential equations using error bounds on path-space observables](https://mlsanity.com/paper/2603.20467/) (2026)

## Abstract (arXiv)

This paper studies the theoretical underpinnings of machine learning of ergodic Ito diffusions. The objective is to understand the convergence properties of the invariant statistics when the underlying system of stochastic differential equations (SDEs) is empirically estimated with a supervised regression framework. Using the perturbation theory of ergodic Markov chains and the linear response theory, we deduce a linear dependence of the errors of one-point and two-point invariant statistics on the error in the learning of the drift and diffusion coefficients. More importantly, our study shows that the usual L2-norm characterization of the learning generalization error is insufficient for achieving this linear dependence result. We find that sufficient conditions for such a linear dependence result are through learning algorithms that produce a uniformly Lipschitz and consistent estimator in the hypothesis space that retains certain characteristics of the drift coefficients, such as the usual linear growth condition that guarantees the existence of solutions of the underlying SDEs. We examine these conditions on two well-understood learning algorithms: the kernel-based spectral regression method and the shallow random neural networks with the ReLU activation function.