# Using Large Ensembles of Control Variates for Variational Inference

arXiv: https://arxiv.org/abs/1810.12482 · HTML: https://mlsanity.com/paper/1810.12482/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2018.

## Built on by (most cited first, 21 of 21)

- [Provable convergence guarantees for black-box variational inference](https://mlsanity.com/paper/2306.03638/) (2023)
- [Efficient Attention via Control Variates](https://mlsanity.com/paper/2302.04542/) (2023)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [Discrete Neural Flow Samplers with Locally Equivariant Transformer](https://mlsanity.com/paper/2505.17741/) (2025)
- [A unified view of likelihood ratio and reparameterization gradients](https://mlsanity.com/paper/2105.14900/) (2021)
- [Better Estimation of the Kullback--Leibler Divergence Between Language Models](https://mlsanity.com/paper/2504.10637/) (2025)
- [Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?](https://mlsanity.com/paper/2307.14642/) (2023)
- [Double Control Variates for Gradient Estimation in Discrete Latent Variable Models](https://mlsanity.com/paper/2111.05300/) (2021)
- [Accelerating Unbiased LLM Evaluation via Synthetic Feedback](https://mlsanity.com/paper/2502.10563/) (2025)
- [Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inference](https://mlsanity.com/paper/2303.10472/) (2023)
- [Sample Average Approximation for Black-Box VI](https://mlsanity.com/paper/2304.06803/) (2023)
- [Variance reduction of diffusion model's gradients with Taylor approximation-based control variate](https://mlsanity.com/paper/2408.12270/) (2024)
- [Neural Control Variates with Automatic Integration](https://mlsanity.com/paper/2409.15394/) (2024)
- [Neural Flow Samplers with Shortcut Models](https://mlsanity.com/paper/2502.07337/) (2025)
- [iMTSP: Solving Min-Max Multiple Traveling Salesman Problem with Imperative Learning](https://mlsanity.com/paper/2405.00285/) (2024)
- [U-Statistics for Importance-Weighted Variational Inference](https://mlsanity.com/paper/2302.13918/) (2023)
- [Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference](https://mlsanity.com/paper/2505.21721/) (2025)
- [Scalable Mean-Field Variational Inference via Preconditioned Primal-Dual Optimization](https://mlsanity.com/paper/2602.07632/) (2026)
- [Joint control variate for faster black-box variational inference](https://mlsanity.com/paper/2210.07290/) (2022)
- [Pathwise Gradient Variance Reduction with Control Variates in Variational Inference](https://mlsanity.com/paper/2410.05753/) (2024)
- [Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space](https://mlsanity.com/paper/2602.18718/) (2026)

## Abstract (arXiv)

Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control variates. Despite the good results obtained, control variates developed for variational inference are typically looked at in isolation. In this paper we clarify the large number of control variates that are available by giving a systematic view of how they are derived. We also present a Bayesian risk minimization framework in which the quality of a procedure for combining control variates is quantified by its effect on optimization convergence rates, which leads to a very simple combination rule. Results show that combining a large number of control variates this way significantly improves the convergence of inference over using the typical gradient estimators or a reduced number of control variates.