# Kernel-based Prediction of Non-Markovian Time Series

arXiv: https://arxiv.org/abs/2007.04286 · HTML: https://mlsanity.com/paper/2007.04286/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2020.

## Built on by (most cited first, 9 of 9)

- [A Framework for Machine Learning of Model Error in Dynamical Systems](https://mlsanity.com/paper/2107.06658/) (2021)
- [Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations](https://mlsanity.com/paper/2108.03561/) (2021)
- [Learning to Forecast Dynamical Systems from Streaming Data](https://mlsanity.com/paper/2109.09703/) (2021)
- [Learning Stochastic Dynamics with Statistics-Informed Neural Network](https://mlsanity.com/paper/2202.12278/) (2022)
- [Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Ito Diffusions](https://mlsanity.com/paper/2105.10102/) (2021)
- [Regression-based projection for learning Mori-Zwanzig operators](https://mlsanity.com/paper/2205.05135/) (2022)
- [Data-driven modelling of nonlinear dynamics by barycentric coordinates and memory](https://mlsanity.com/paper/2112.06742/) (2021)
- [Ensemble forecasts in reproducing kernel Hilbert space family](https://mlsanity.com/paper/2207.14653/) (2022)
- [Data-driven Mori-Zwanzig modeling of Lagrangian particle dynamics in turbulent flows](https://mlsanity.com/paper/2507.16058/) (2025)

## Abstract (arXiv)

A nonparametric method to predict non-Markovian time series of partially observed dynamics is developed. The prediction problem we consider is a supervised learning task of finding a regression function that takes a delay embedded observable to the observable at a future time. When delay embedding theory is applicable, the proposed regression function is a consistent estimator of the flow map induced by the delay embedding. Furthermore, the corresponding Mori-Zwanzig equation governing the evolution of the observable simplifies to only a Markovian term, represented by the regression function. We realize this supervised learning task with a class of kernel-based linear estimators, the kernel analog forecast (KAF), which are consistent in the limit of large data. In a scenario with a high-dimensional covariate space, we employ a Markovian kernel smoothing method which is computationally cheaper than the Nystr"om projection method for realizing KAF. In addition to the guaranteed theoretical convergence, we numerically demonstrate the effectiveness of this approach on higher-dimensional problems where the relevant kernel features are difficult to capture with the Nystr"om method. Given noisy training data, we propose a nonparametric smoother as a de-noising method. Numerically, we show that the proposed smoother is more accurate than EnKF and 4Dvar in de-noising signals corrupted by independent (but not necessarily identically distributed) noise, even if the smoother is constructed using a data set corrupted by white noise. We show skillful prediction using the KAF constructed from the denoised data.