# Lightweight Gaussian Process Inference in C++ on Metal and CUDA

arXiv: https://arxiv.org/abs/2605.17898 · HTML: https://mlsanity.com/paper/2605.17898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 18 May 2026.

## Builds on (most cited first, 5 of 5)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Kernel Operations on the GPU, with Autodiff, without Memory Overflows](https://mlsanity.com/paper/2004.11127/) (2020)
- [Constant-Time Predictive Distributions for Gaussian Processes](https://mlsanity.com/paper/1803.06058/) (2018)

## Abstract (arXiv)

Gaussian process (GP) inference in Python is dominated by libraries such as GPyTorch and GPflow, which are built on deep-learning frameworks and inherit their dispatch overhead and dependency footprint. We present LightGP, a dependency-free C++17 library for GP regression with Python bindings, supporting Apple Metal and NVIDIA CUDA backends alongside tuned CPU paths via Apple Accelerate and OpenBLAS. LightGP provides four inference paths -- exact Cholesky, matrix-free conjugate gradients, sparse variational free energy, and structured kernel interpolation with FFT -- covering problems from N=100 to N=500,000. On an Apple M4, LightGP CPU is 2.6--8.7× faster than GPyTorch CPU for exact GP and ~1.5× faster for sparse GP at every scale tested. On an NVIDIA RTX~3060, LightGP CUDA is 2.3--6.7× faster than GPyTorch CUDA for exact GP up to N=2,048, with GPyTorch closing the gap at N=4,096. A fused matrix-free kernel-vector product on Metal achieves 32× over the explicit path at N=20,000 with O(N) memory, and an FFT-accelerated SKI matvec via Accelerate vDSP runs in sub-millisecond time at N=200,000. LightGP compiles as a single static library with zero external dependencies and is installable via textttpip install lightgp