# Beyond State Space Representation: A General Theory for Kernel Packets

arXiv: https://arxiv.org/abs/2402.04022 · HTML: https://mlsanity.com/paper/2402.04022/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 13 of 13)

- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Variational Fourier features for Gaussian processes](https://mlsanity.com/paper/1611.06740/) (2016)
- [Fastfood: Approximate Kernel Expansions in Loglinear Time](https://mlsanity.com/paper/1408.3060/) (2014)
- [A general framework for Vecchia approximations of Gaussian processes](https://mlsanity.com/paper/1708.06302/) (2017)
- [Local Gaussian process approximation for large computer experiments](https://mlsanity.com/paper/1303.0383/) (2013)
- [Linear-time inference for Gaussian Processes on one dimension](https://mlsanity.com/paper/2003.05554/) (2020)
- [Locally induced Gaussian processes for large-scale simulation experiments](https://mlsanity.com/paper/2008.12857/) (2020)
- [Kernel Packet: An Exact and Scalable Algorithm for Gaussian Process Regression with Mat'ern Correlations](https://mlsanity.com/paper/2203.03116/) (2022)
- [Fast prediction of deterministic functions using sparse grid experimental designs](https://mlsanity.com/paper/1402.6350/) (2014)
- [Sample and Computationally Efficient Stochastic Kriging in High Dimensions](https://mlsanity.com/paper/2010.06802/) (2020)
- [Random Smoothing Regularization in Kernel Gradient Descent Learning](https://mlsanity.com/paper/2305.03531/) (2023)
- [Linear-Cost Covariance Functions for Gaussian Random Fields](https://mlsanity.com/paper/1711.05895/) (2017)
- [State Space representation of non-stationary Gaussian Processes](https://mlsanity.com/paper/1601.01544/) (2016)

## Abstract (arXiv)

Gaussian process (GP) regression provides a flexible, nonparametric framework for probabilistic modeling, yet remains computationally demanding in large-scale applications. For one-dimensional data, state space (SS) models achieve linear-time inference by reformulating GPs as stochastic differential equations (SDEs). However, SS approaches are confined to gridded inputs and cannot handle multi-dimensional scattered data. We propose a new framework based on kernel packet (KP), which overcomes these limitations while retaining exactness and scalability. A KP is a compactly supported function defined as a linear combination of the GP covariance functions. In this article, we prove that KPs can be identified via the forward and backward SS representations. We also show that the KP approach enables exact inference with linear-time training and logarithmic or constant-time prediction, and extends naturally to multi-dimensional gridded or scattered data without low-rank approximations. Numerical experiments on large-scale additive and product-form GPs with millions of samples demonstrate that KPs achieve exact, memory-efficient inference where SDE-based and low-rank GP methods fail.