# Meta-learning Control Variates: Variance Reduction with Limited Data

arXiv: https://arxiv.org/abs/2303.04756 · HTML: https://mlsanity.com/paper/2303.04756/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 71 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [Survey of multifidelity methods in uncertainty propagation, inference, and optimization](https://mlsanity.com/paper/1806.10761/) (2018)
- [Online Meta-Learning](https://mlsanity.com/paper/1902.08438/) (2019)
- [On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms](https://mlsanity.com/paper/1908.10400/) (2019)
- [Backpropagation through the Void: Optimizing control variates for black-box gradient estimation](https://mlsanity.com/paper/1711.00123/) (2017)
- [Generalized Inner Loop Meta-Learning](https://mlsanity.com/paper/1910.01727/) (2019)
- [Control functionals for Monte Carlo integration](https://mlsanity.com/paper/1410.2392/) (2014)
- [Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis](https://mlsanity.com/paper/2109.14595/) (2021)
- [Neural Control Variates](https://mlsanity.com/paper/2006.01524/) (2020)
- [Stein's Method Meets Computational Statistics: A Review of Some Recent Developments](https://mlsanity.com/paper/2105.03481/) (2021)
- [Approximate Inference for Fully Bayesian Gaussian Process Regression](https://mlsanity.com/paper/1912.13440/) (2019)
- [Control Variates for Stochastic Gradient MCMC](https://mlsanity.com/paper/1706.05439/) (2017)
- [Gradient Estimation with Discrete Stein Operators](https://mlsanity.com/paper/2202.09497/) (2022)
- [Active Multi-Information Source Bayesian Quadrature](https://mlsanity.com/paper/1903.11331/) (2019)
- [Bayesian Quadrature for Multiple Related Integrals](https://mlsanity.com/paper/1801.04153/) (2018)
- [Vector-Valued Control Variates](https://mlsanity.com/paper/2109.08944/) (2021)
- [Regularized Zero-Variance Control Variates](https://mlsanity.com/paper/1811.05073/) (2018)
- [Scalable Control Variates for Monte Carlo Methods via Stochastic Optimization](https://mlsanity.com/paper/2006.07487/) (2020)
- [Semi-Exact Control Functionals From Sard's Method](https://mlsanity.com/paper/2002.00033/) (2020)
- [Multilevel Bayesian Quadrature](https://mlsanity.com/paper/2210.08329/) (2022)
- [Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayes](https://mlsanity.com/paper/2302.11709/) (2023)

## Built on by (most cited first, 7 of 7)

- [Conditional Bayesian Quadrature](https://mlsanity.com/paper/2406.16530/) (2024)
- [Theoretical guarantees for neural control variates in MCMC](https://mlsanity.com/paper/2304.01111/) (2023)
- [Bayesian Numerical Integration with Neural Networks](https://mlsanity.com/paper/2305.13248/) (2023)
- [Conditional neural control variates for variance reduction in Bayesian inverse problems](https://mlsanity.com/paper/2602.21357/) (2026)
- [Training neural control variates using correlated configurations](https://mlsanity.com/paper/2505.07719/) (2025)
- [Amortized quadrature for posterior expectations in inverse problems](https://mlsanity.com/paper/2606.15871/) (2026)
- [Amortised adaptive importance sampling](https://mlsanity.com/paper/2610.16736/) (2026)

## Abstract (arXiv)

Control variates can be a powerful tool to reduce the variance of Monte Carlo estimators, but constructing effective control variates can be challenging when the number of samples is small. In this paper, we show that when a large number of related integrals need to be computed, it is possible to leverage the similarity between these integration tasks to improve performance even when the number of samples per task is very small. Our approach, called meta learning CVs (Meta-CVs), can be used for up to hundreds or thousands of tasks. Our empirical assessment indicates that Meta-CVs can lead to significant variance reduction in such settings, and our theoretical analysis establishes general conditions under which Meta-CVs can be successfully trained.