mlsanityData to 7 Oct 2026

Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models

arXiv 1803.09151 ↗First posted 24 Mar 2018stat.ML cs.LGcited by 38

Not on the map. Why?

Abstract (arXiv)

The natural gradient method has been used effectively in conjugate Gaussian process models, but the non-conjugate case has been largely unexplored. We examine how natural gradients can be used in non-conjugate stochastic settings, together with hyperparameter learning. We conclude that the natural gradient can significantly improve performance in terms of wall-clock time. For ill-conditioned posteriors the benefit of the natural gradient method is especially pronounced, and we demonstrate a practical setting where ordinary gradients are unusable. We show how natural gradients can be computed efficiently and automatically in any parameterization, using automatic differentiation. Our code is integrated into the GPflow package.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 5; 2023: 8; 2024: 5; 2025: 5; 2026: 1.

38 papers.

Sort:

  1. 2021The Bayesian Learning Rulecited by 95
  2. 2021Spatio-Temporal Variational Gaussian Processescited by 32
  3. 2021Tractable structured natural gradient descent using local parameterizationscited by 22
  4. 2022A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Modelscited by 22
  5. 2021Dual Parameterization of Sparse Variational Gaussian Processescited by 19
  6. 2023The Memory Perturbation Equation: Understanding Model's Sensitivity to Datacited by 16
  7. 2022Partitioned Variational Inference: A Framework for Probabilistic Federated Learningcited by 13
  8. 2021GPflux: A Library for Deep Gaussian Processescited by 12
  9. 2021Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guaranteescited by 12
  10. 2021Scalable Gaussian Processes for Data-Driven Design using Big Data with Categorical Factorscited by 11
  11. 2021Sparse Algorithms for Markovian Gaussian Processescited by 10
  12. 2025Epistemic Uncertainty in Conformal Scores: A Unified Approachcited by 9
26 more
  1. 2022Variational Nearest Neighbor Gaussian Processcited by 8
  2. 2024Understanding Stochastic Natural Gradient Variational Inferencecited by 8
  3. 2024Physics-Informed Variational State-Space Gaussian Processescited by 7
  4. 2021Hierarchical Inducing Point Gaussian Process for Inter-domain Observationscited by 6
  5. 2021Multivariate Probabilistic Regression with Natural Gradient Boostingcited by 5
  6. 2021Input Dependent Sparse Gaussian Processescited by 5
  7. 2023Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trainscited by 5
  8. 2024Continuous Bayesian Model Selection for Multivariate Causal Discoverycited by 5
  9. 2021Gaussian Processes on Hypergraphscited by 4
  10. 2025SING: SDE Inference via Natural Gradientscited by 4
  11. 2023Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Modelscited by 3
  12. 2023Convolutional Deep Kernel Machinescited by 3
  13. 2025Optimization Guarantees for Square-Root Natural-Gradient Variational Inferencecited by 3
  14. 2025Natural Gradient VI: Guarantees for Non-Conjugate Modelscited by 3
  15. 2022DPVIm: Differentially Private Variational Inference Improvedcited by 2
  16. 2023A Unifying Variational Framework for Gaussian Process Motion Planningcited by 2
  17. 2023Bayesian Analysis of Combinatorial Gaussian Process Banditscited by 2
  18. 2024Sparse Variational Student-t Processes for Heavy-tailed Modelingcited by 2
  19. 2021Variational Inference at Glacier Scalecited by 1
  20. 2023Optimising Distributions with Natural Gradient Surrogatescited by 1
  21. 2024Accelerating Convergence in Bayesian Few-Shot Classificationcited by 1
  22. 2021Searching in the Forest for Local Bayesian Optimizationcited by 0
  23. 2022Fast emulation of density functional theory simulations using approximate Gaussian processescited by 0
  24. 2023Function-Space Regularization for Deep Bayesian Classificationcited by 0
  25. 2025Spectral-factorized Positive-definite Curvature Learning for NN Trainingcited by 0
  26. 2026Inverse-Free Sparse Variational Gaussian Processescited by 0