mlsanityData to 7 Oct 2026

Bias-Free Scalable Gaussian Processes via Randomized Truncations

arXiv 2102.06695 ↗First posted 12 Feb 2021cs.LG stat.MLcited by 15

Not on the map. Why?

Abstract (arXiv)

Scalable Gaussian Process methods are computationally attractive, yet introduce modeling biases that require rigorous study. This paper analyzes two common techniques: early truncated conjugate gradients (CG) and random Fourier features (RFF). We find that both methods introduce a systematic bias on the learned hyperparameters: CG tends to underfit while RFF tends to overfit. We address these issues using randomized truncation estimators that eliminate bias in exchange for increased variance. In the case of RFF, we show that the bias-to-variance conversion is indeed a trade-off: the additional variance proves detrimental to optimization. However, in the case of CG, our unbiased learning procedure meaningfully outperforms its biased counterpart with minimal additional computation.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 12 Feb 2021.

Builds on

21 of 41 references.

References · 21

  1. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  2. 2013Gaussian Processes for Big Datacited by 377
  3. 2019Residual Flows for Invertible Generative Modelingcited by 256
  4. 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 125
  5. 2019Exact Gaussian Processes on a Million Data Pointscited by 118
  6. 2015Distributed Gaussian Processescited by 84
  7. 2015On the Error of Random Fourier Featurescited by 76
  8. 2014Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasetscited by 71
  9. 2016Understanding Probabilistic Sparse Gaussian Process Approximationscited by 68
  10. 2014Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictionscited by 66
  11. 2017A general framework for Vecchia approximations of Gaussian processescited by 66
  12. 2019Rates of Convergence for Sparse Variational Gaussian Process Regressioncited by 62
9 more
  1. 2018Constant-Time Predictive Distributions for Gaussian Processescited by 48
  2. 2017Scalable Log Determinants for Gaussian Process Kernel Learningcited by 34
  3. 2016Preconditioning Kernel Matricescited by 32
  4. 2020Randomized Automatic Differentiationcited by 23
  5. 2013On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoodscited by 16
  6. 2020SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Modelscited by 14
  7. 2020Linear-time inference for Gaussian Processes on one dimensioncited by 11
  8. 2019Efficient Optimization of Loops and Limits with Randomized Telescoping Sumscited by 7
  9. 2015Enabling scalable stochastic gradient-based inference for Gaussian processes by employing the Unbiased LInear System SolvEr (ULISSE)cited by 3

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 4; 2023: 1; 2024: 3; 2025: 1.

15 papers.

Sort:

  1. 2021Rectangular Flows for Manifold Learningcited by 49
  2. 2022Posterior and Computational Uncertainty in Gaussian Processescited by 23
  3. 2021Preconditioning for Scalable Gaussian Process Hyperparameter Optimizationcited by 22
  4. 2021When are Iterative Gaussian Processes Reliably Accurate?cited by 12
  5. 2023Large-Scale Gaussian Processes via Alternating Projectioncited by 11
  6. 2021SKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processescited by 9
  7. 2022Variational Nearest Neighbor Gaussian Processcited by 8
  8. 2024Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inferencecited by 8
  9. 2022Neural Implicit Manifold Learning for Topology-Aware Density Estimationcited by 5
  10. 2022Multi-fidelity Monte Carlo: a pseudo-marginal approachcited by 4
  11. 2021Scaling Structured Inference with Randomizationcited by 2
  12. 2024Recommendations for Baselines and Benchmarking Approximate Gaussian Processescited by 2
3 more
  1. 2025Learning Hyperparameters via a Data-Emphasized Variational Objectivecited by 2
  2. 2021Barely Biased Learning for Gaussian Process Regressioncited by 0
  3. 2024Kernel Multigrid: Accelerate Back-fitting via Sparse Gaussian Process Regressioncited by 0