# Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition

arXiv: https://arxiv.org/abs/1710.07324 · HTML: https://mlsanity.com/paper/1710.07324/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2017.

## Built on by (most cited first, 28 of 28)

- [End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes](https://mlsanity.com/paper/2305.15930/) (2023)
- [GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning](https://mlsanity.com/paper/2102.07868/) (2021)
- [Posterior and Computational Uncertainty in Gaussian Processes](https://mlsanity.com/paper/2205.15449/) (2022)
- [Large-Scale Learning with Fourier Features and Tensor Decompositions](https://mlsanity.com/paper/2109.01545/) (2021)
- [Solving High Frequency and Multi-Scale PDEs with Gaussian Processes](https://mlsanity.com/paper/2311.04465/) (2023)
- [Permutation Search of Tensor Network Structures via Local Sampling](https://mlsanity.com/paper/2206.06597/) (2022)
- [Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling](https://mlsanity.com/paper/2205.14283/) (2022)
- [Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluations](https://mlsanity.com/paper/2304.12875/) (2023)
- [Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processing](https://mlsanity.com/paper/2203.06031/) (2022)
- [SKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processes](https://mlsanity.com/paper/2106.06695/) (2021)
- [Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition](https://mlsanity.com/paper/2106.05992/) (2021)
- [tntorch: Tensor Network Learning with PyTorch](https://mlsanity.com/paper/2206.11128/) (2022)
- [Toward Efficient Kernel-Based Solvers for Nonlinear PDEs](https://mlsanity.com/paper/2410.11165/) (2024)
- [Hierarchical Inducing Point Gaussian Process for Inter-domain Observations](https://mlsanity.com/paper/2103.00393/) (2021)
- [tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)](https://mlsanity.com/paper/2402.02456/) (2024)
- [Position: Tensor Networks are a Valuable Asset for Green AI](https://mlsanity.com/paper/2205.12961/) (2022)
- [Tensor Network-Constrained Kernel Machines as Gaussian Processes](https://mlsanity.com/paper/2403.19500/) (2024)
- [Interpretable Bayesian Tensor Network Kernel Machines with Automatic Rank and Feature Selection](https://mlsanity.com/paper/2507.11136/) (2025)
- [Kernel Interpolation with Sparse Grids](https://mlsanity.com/paper/2305.14451/) (2023)
- [Projecting basis functions with tensor networks for Gaussian process regression](https://mlsanity.com/paper/2310.20630/) (2023)
- [Lower and Upper Bounds on the VC-Dimension of Tensor Network Models](https://mlsanity.com/paper/2106.11827/) (2021)
- [Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs](https://mlsanity.com/paper/2510.13772/) (2025)
- [Function-Space Regularization for Deep Bayesian Classification](https://mlsanity.com/paper/2307.06055/) (2023)
- [Exploiting Hankel-Toeplitz Structures for Fast Computation of Kernel Precision Matrices](https://mlsanity.com/paper/2408.02346/) (2024)
- [Adaptive Basis Function Selection for Computationally Efficient Predictions](https://mlsanity.com/paper/2408.07480/) (2024)
- [Tensor network square root Kalman filter for online Gaussian process regression](https://mlsanity.com/paper/2409.03276/) (2024)
- [Kronecker-Structured Nonparametric Spatiotemporal Point Processes](https://mlsanity.com/paper/2603.23746/) (2026)
- [Scalable Gaussian process inference via neural feature maps](https://mlsanity.com/paper/2605.10285/) (2026)

## Abstract (arXiv)

We propose a method (TT-GP) for approximate inference in Gaussian Process (GP) models. We build on previous scalable GP research including stochastic variational inference based on inducing inputs, kernel interpolation, and structure exploiting algebra. The key idea of our method is to use Tensor Train decomposition for variational parameters, which allows us to train GPs with billions of inducing inputs and achieve state-of-the-art results on several benchmarks. Further, our approach allows for training kernels based on deep neural networks without any modifications to the underlying GP model. A neural network learns a multidimensional embedding for the data, which is used by the GP to make the final prediction. We train GP and neural network parameters end-to-end without pretraining, through maximization of GP marginal likelihood. We show the efficiency of the proposed approach on several regression and classification benchmark datasets including MNIST, CIFAR-10, and Airline.