mlsanityData to 7 Oct 2026

Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models

arXiv 1703.04265 ↗First posted 13 Mar 2017cs.LGcited by 65

Not on the map. Why?

Abstract (arXiv)

Variational inference is computationally challenging in models that contain both conjugate and non-conjugate terms. Methods specifically designed for conjugate models, even though computationally efficient, find it difficult to deal with non-conjugate terms. On the other hand, stochastic-gradient methods can handle the non-conjugate terms but they usually ignore the conjugate structure of the model which might result in slow convergence. In this paper, we propose a new algorithm called Conjugate-computation Variational Inference (CVI) which brings the best of the two worlds together -- it uses conjugate computations for the conjugate terms and employs stochastic gradients for the rest. We derive this algorithm by using a stochastic mirror-descent method in the mean-parameter space, and then expressing each gradient step as a variational inference in a conjugate model. We demonstrate our algorithm's applicability to a large class of models and establish its convergence. Our experimental results show that our method converges much faster than the methods that ignore the conjugate structure of the model.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 10; 2023: 16; 2024: 13; 2025: 13; 2026: 2.

65 papers.

Sort:

By area:

  1. 2021The Bayesian Learning Rulecited by 95
  2. 2024Variational Learning is Effective for Large Deep Networkscited by 62
  3. 2021Spatio-Temporal Variational Gaussian Processescited by 32
  4. 2024Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspectivecited by 25
  5. 2021Conditioning Sparse Variational Gaussian Processes for Online Decision-makingcited by 23
  6. 2021Tractable structured natural gradient descent using local parameterizationscited by 22
  7. 2022A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Modelscited by 22
  8. 2023On the Convergence of Black-Box Variational Inferencecited by 22
  9. 2023Sampling via Gradient Flows in the Space of Probability Measurescited by 20
  10. 2021Dual Parameterization of Sparse Variational Gaussian Processescited by 19
  11. 2022Markovian Gaussian Process Variational Autoencoderscited by 16
  12. 2023Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariancecited by 16
38 more
  1. 2023The Memory Perturbation Equation: Understanding Model's Sensitivity to Datacited by 16
  2. 2024Bayesian Online Natural Gradient (BONG)cited by 14
  3. 2022Partitioned Variational Inference: A Framework for Probabilistic Federated Learningcited by 13
  4. 2025Understanding and Improving Shampoo and SOAP via Kullback-Leibler Minimizationcited by 13
  5. 2021Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guaranteescited by 12
  6. 2022Bayesian Continual Learning via Spiking Neural Networkscited by 12
  7. 2023Revisiting Structured Variational Autoencoderscited by 11
  8. 2021A practical tutorial on Variational Bayescited by 10
  9. 2021Sparse Algorithms for Markovian Gaussian Processescited by 10
  10. 2023Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learningcited by 10
  11. 2023Variational Gaussian Process Diffusion Processescited by 10
  12. 2023Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFACcited by 9
  13. 2024Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flowscited by 9
  14. 2021Bayesian Variational Federated Learning and Unlearning in Decentralized Networkscited by 8
  15. 2024Understanding Stochastic Natural Gradient Variational Inferencecited by 8
  16. 2021Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processescited by 7
  17. 2022Fantasizing with Dual GPs in Bayesian Optimization and Active Learningcited by 7
  18. 2024eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modelingcited by 7
  19. 2024Function-space Parameterization of Neural Networks for Sequential Learningcited by 7
  20. 2024Physics-Informed Variational State-Space Gaussian Processescited by 7
  21. 2025Knowledge Adaptation as Posterior Correctioncited by 6
  22. 2021BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rulecited by 5
  23. 2022Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics in Limit-Order Book Marketscited by 5
  24. 2023Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trainscited by 5
  25. 2025Connecting Federated ADMM to Bayescited by 4
  26. 2025SING: SDE Inference via Natural Gradientscited by 4
  27. 2022Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learningcited by 3
  28. 2023The Lie-Group Bayesian Learning Rulecited by 3
  29. 2023Real-Time Variational Method for Learning Neural Trajectory and its Dynamicscited by 3
  30. 2023Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Modelscited by 3
  31. 2025Federated ADMM from Bayesian Dualitycited by 3
  32. 2025Optimization Guarantees for Square-Root Natural-Gradient Variational Inferencecited by 3
  33. 2025Natural Gradient VI: Guarantees for Non-Conjugate Modelscited by 3
  34. 2022Differentially private partitioned variational inferencecited by 2
  35. 2025Information Geometry of Variational Bayescited by 2
  36. 2022Manifold Gaussian Variational Bayes on the Precision Matrixcited by 1
  37. 2023Short-term Prediction and Filtering of Solar Power Using State-Space Gaussian Processescited by 1
  38. 2023Variational Bayes Made Easycited by 1

50 most cited of 65 All 65 (JSON)