# When can Regression-Adjusted Control Variates Help? Rare Events, Sobolev Embedding and Minimax Optimality

arXiv: https://arxiv.org/abs/2305.16527 · HTML: https://mlsanity.com/paper/2305.16527/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 73 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 24 of 24)

- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions](https://mlsanity.com/paper/1502.06800/) (2015)
- [Hutch++: Optimal Stochastic Trace Estimation](https://mlsanity.com/paper/2010.09649/) (2020)
- [Control functionals for Monte Carlo integration](https://mlsanity.com/paper/1410.2392/) (2014)
- [Kernel Thinning](https://mlsanity.com/paper/2105.05842/) (2021)
- [Stein Points](https://mlsanity.com/paper/1803.10161/) (2018)
- [Neural Control Variates](https://mlsanity.com/paper/2006.01524/) (2020)
- [Generalized Kernel Thinning](https://mlsanity.com/paper/2110.01593/) (2021)
- [Kernel quadrature with DPPs](https://mlsanity.com/paper/1906.07832/) (2019)
- [Positively Weighted Kernel Quadrature via Subsampling](https://mlsanity.com/paper/2107.09597/) (2021)
- [Gradient Estimation with Discrete Stein Operators](https://mlsanity.com/paper/2202.09497/) (2022)
- [Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering](https://mlsanity.com/paper/1501.02056/) (2015)
- [Monte Carlo with Determinantal Point Processes](https://mlsanity.com/paper/1605.00361/) (2016)
- [Convergence Guarantees for Adaptive Bayesian Quadrature Methods](https://mlsanity.com/paper/1905.10271/) (2019)
- [Sampling-based Nystr"om Approximation and Kernel Quadrature](https://mlsanity.com/paper/2301.09517/) (2023)
- [Optimal rates of entropy estimation over Lipschitz balls](https://mlsanity.com/paper/1711.02141/) (2017)
- [Convergence guarantees for kernel-based quadrature rules in misspecified settings](https://mlsanity.com/paper/1605.07254/) (2016)
- [Regularized Zero-Variance Control Variates](https://mlsanity.com/paper/1811.05073/) (2018)
- [Convergence Rates for Non-Log-Concave Sampling and Log-Partition Estimation](https://mlsanity.com/paper/2303.03237/) (2023)
- [Nonparametric Estimation of Renyi Divergence and Friends](https://mlsanity.com/paper/1402.2966/) (2014)
- [Theoretical guarantees for neural control variates in MCMC](https://mlsanity.com/paper/2304.01111/) (2023)
- [On Estimation of Lr-Norms in Gaussian White Noise Models](https://mlsanity.com/paper/1710.03863/) (2017)
- [Approximate Euclidean lengths and distances beyond Johnson-Lindenstrauss](https://mlsanity.com/paper/2205.12307/) (2022)

## Built on by (most cited first, 4 of 4)

- [Regression for the Mean: Auto-Evaluation and Inference with Few Labels through Post-hoc Regression](https://mlsanity.com/paper/2411.12665/) (2024)
- [Nearest Neighbor Sampling for Covariate Shift Adaptation](https://mlsanity.com/paper/2312.09969/) (2023)
- [Prediction-Enhanced Monte Carlo: A Machine Learning View on Control Variate](https://mlsanity.com/paper/2412.11257/) (2024)
- [Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift](https://mlsanity.com/paper/2506.23453/) (2025)

## Abstract (arXiv)

This paper studies the use of a machine learning-based estimator as a control variate for mitigating the variance of Monte Carlo sampling. Specifically, we seek to uncover the key factors that influence the efficiency of control variates in reducing variance. We examine a prototype estimation problem that involves simulating the moments of a Sobolev function based on observations obtained from (random) quadrature nodes. Firstly, we establish an information-theoretic lower bound for the problem. We then study a specific quadrature rule that employs a nonparametric regression-adjusted control variate to reduce the variance of the Monte Carlo simulation. We demonstrate that this kind of quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under a sufficient smoothness assumption. Due to the Sobolev Embedding Theorem, the sufficient smoothness assumption eliminates the existence of rare and extreme events. Finally, we show that, in the presence of rare and extreme events, a truncated version of the Monte Carlo algorithm can achieve the minimax optimal rate while the control variate cannot improve the convergence rate.