# Empirical Bayes Rebiasing

arXiv: https://arxiv.org/abs/2605.08069 · HTML: https://mlsanity.com/paper/2605.08069/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 68 references that are in the map.
- First posted on arXiv 8 May 2026.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy](https://mlsanity.com/paper/2507.01352/) (2025)
- [PPI++: Efficient Prediction-Powered Inference](https://mlsanity.com/paper/2311.01453/) (2023)
- [Prediction-Powered Ranking of Large Language Models](https://mlsanity.com/paper/2402.17826/) (2024)
- [Combining Observational and Experimental Datasets Using Shrinkage Estimators](https://mlsanity.com/paper/2002.06708/) (2020)
- [Prediction-Powered Inference with Imputed Covariates and Nonuniform Sampling](https://mlsanity.com/paper/2501.18577/) (2025)
- [Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models](https://mlsanity.com/paper/2008.08244/) (2020)
- [Data enriched linear regression](https://mlsanity.com/paper/1304.1837/) (2013)
- [Prediction-Powered Adaptive Shrinkage Estimation](https://mlsanity.com/paper/2502.14166/) (2025)
- [Covariate-Powered Empirical Bayes Estimation](https://mlsanity.com/paper/1906.01611/) (2019)
- [More power to you: Using machine learning to augment human coding for more efficient inference in text-based randomized trials](https://mlsanity.com/paper/2309.13666/) (2023)
- [Sensitivity Analysis using Approximate Moment Condition Models](https://mlsanity.com/paper/1808.07387/) (2018)
- [Robust Empirical Bayes Confidence Intervals](https://mlsanity.com/paper/2004.03448/) (2020)
- [Slamming the sham: A Bayesian model for adaptive adjustment with noisy control data](https://mlsanity.com/paper/1905.09693/) (2019)
- [Empirical Bayes Double Shrinkage for Combining Biased and Unbiased Causal Estimates](https://mlsanity.com/paper/2309.06727/) (2023)
- [(Empirical) Bayes Approaches to Parallel Trends](https://mlsanity.com/paper/2404.11839/) (2024)
- [Introducing the b-value: combining unbiased and biased estimators from a sensitivity analysis perspective](https://mlsanity.com/paper/2602.16310/) (2026)
- [The Illusion of Learning from Observational Data: An Empirical Bayes Perspective](https://mlsanity.com/paper/2604.08853/) (2026)

## Abstract (arXiv)

We study methods for simultaneous analysis of many noisy and biased estimates, each paired with an even noisier estimate of its own bias. The analyst's goal is to construct short calibrated intervals for each parameter. The standard debiasing approach, which subtracts the bias estimate from each biased estimate, inflates variance and yields long intervals. In this paper, we propose an empirical Bayes rebiasing strategy that starts from the fully debiased estimates and learns from data how much bias to reintroduce by estimating the unknown bias distribution. We provide convergence rates for the coverage of our intervals when the bias distribution is estimated using nonparametric maximum likelihood. Furthermore, we demonstrate substantial precision gains in prediction-powered inference, including pairwise LLM win-rate evaluations, as well as for inference of direct genetic effects in family-based GWAS.