# Entropy Regularization for Population Estimation

arXiv: https://arxiv.org/abs/2208.11747 · HTML: https://mlsanity.com/paper/2208.11747/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 24 Aug 2022.

## Builds on (most cited first, 23 of 23)

- [Doubly Robust Off-policy Value Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1511.03722/) (2015)
- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [Understanding the impact of entropy on policy optimization](https://mlsanity.com/paper/1811.11214/) (2018)
- [Doubly Robust Policy Evaluation and Optimization](https://mlsanity.com/paper/1503.02834/) (2015)
- [Tsallis-INF: An Optimal Algorithm for Stochastic and Adversarial Bandits](https://mlsanity.com/paper/1807.07623/) (2018)
- [Inference for Batched Bandits](https://mlsanity.com/paper/2002.03217/) (2020)
- [Accurate Inference for Adaptive Linear Models](https://mlsanity.com/paper/1712.06695/) (2017)
- [Why Adaptively Collected Data Have Negative Bias and How to Correct for It](https://mlsanity.com/paper/1708.01977/) (2017)
- [Post-Contextual-Bandit Inference](https://mlsanity.com/paper/2106.00418/) (2021)
- [Balanced Linear Contextual Bandits](https://mlsanity.com/paper/1812.06227/) (2018)
- [Power Constrained Bandits](https://mlsanity.com/paper/2004.06230/) (2020)
- [On the bias, risk and consistency of sample means in multi-armed bandits](https://mlsanity.com/paper/1902.00746/) (2019)
- [Are sample means in multi-armed bandits positively or negatively biased?](https://mlsanity.com/paper/1905.11397/) (2019)
- [Online Multi-Armed Bandits with Adaptive Inference](https://mlsanity.com/paper/2102.13202/) (2021)
- [Mitigating Bias in Adaptive Data Gathering via Differential Privacy](https://mlsanity.com/paper/1806.02329/) (2018)
- [Your Policy Regularizer is Secretly an Adversary](https://mlsanity.com/paper/2203.12592/) (2022)
- [Fair Algorithms for Infinite and Contextual Bandits](https://mlsanity.com/paper/1610.09559/) (2016)
- [Online Debiasing for Adaptively Collected High-dimensional Data with Applications to Time Series Analysis](https://mlsanity.com/paper/1911.01040/) (2019)
- [On conditional versus marginal bias in multi-armed bandits](https://mlsanity.com/paper/2002.08422/) (2020)
- [Beyond Ads: Sequential Decision-Making Algorithms in Law and Public Policy](https://mlsanity.com/paper/2112.06833/) (2021)
- [Integrating Reward Maximization and Population Estimation: Sequential Decision-Making for Internal Revenue Service Audit Selection](https://mlsanity.com/paper/2204.11910/) (2022)
- [Regularized Contextual Bandits](https://mlsanity.com/paper/1810.05065/) (2018)
- [Reconciling Risk Allocation and Prevalence Estimation in Public Health Using Batched Bandits](https://mlsanity.com/paper/2110.13306/) (2021)

## Built on by (most cited first, 1 of 1)

- [Auditing Fairness by Betting](https://mlsanity.com/paper/2305.17570/) (2023)

## Abstract (arXiv)

Entropy regularization is known to improve exploration in sequential decision-making problems. We show that this same mechanism can also lead to nearly unbiased and lower-variance estimates of the mean reward in the optimize-and-estimate structured bandit setting. Mean reward estimation (i.e., population estimation) tasks have recently been shown to be essential for public policy settings where legal constraints often require precise estimates of population metrics. We show that leveraging entropy and KL divergence can yield a better trade-off between reward and estimator variance than existing baselines, all while remaining nearly unbiased. These properties of entropy regularization illustrate an exciting potential for bridging the optimal exploration and estimation literatures.