# A Flexible Defense Against the Winner's Curse

arXiv: https://arxiv.org/abs/2411.18569 · HTML: https://mlsanity.com/paper/2411.18569/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Builds on (most cited first, 11 of 11)

- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [A Computational Approach to Politeness with Application to Social Factors](https://mlsanity.com/paper/1306.6078/) (2013)
- [Exact post-selection inference, with application to the lasso](https://mlsanity.com/paper/1311.6238/) (2013)
- [Optimal Inference After Model Selection](https://mlsanity.com/paper/1410.2597/) (2014)
- [Splitting strategies for post-selection inference](https://mlsanity.com/paper/2102.02159/) (2021)
- [Data fission: splitting a single data point](https://mlsanity.com/paper/2112.11079/) (2021)
- [Data thinning for convolution-closed distributions](https://mlsanity.com/paper/2301.07276/) (2023)
- [Post-Selection Inference via Algorithmic Stability](https://mlsanity.com/paper/2011.09462/) (2020)
- [Locally Simultaneous Inference](https://mlsanity.com/paper/2212.09009/) (2022)
- [Confidence Intervals for Selected Parameters](https://mlsanity.com/paper/1906.00505/) (2019)

## Built on by (most cited first, 7 of 7)

- [Prompt-Dependent Ranking of Large Language Models with Uncertainty Quantification](https://mlsanity.com/paper/2603.03336/) (2026)
- [Winner's Curse Drives False Promises in Data-Driven Decisions: A Case Study in Refugee Matching](https://mlsanity.com/paper/2602.08892/) (2026)
- [Robustness of Refugee-Matching Gains to Off-Policy Evaluation Choices](https://mlsanity.com/paper/2605.06686/) (2026)
- [Bayesian Inference and Decision Audits for Public Archives of Frontier AI Evaluations](https://mlsanity.com/paper/2606.17005/) (2026)
- [How Sensitive Are LLM Leaderboard Claims to Hidden Model Selection?](https://mlsanity.com/paper/2609.28177/) (2026)
- [Prequential E-Values for Selected-GP Near-Optimality Certificates](https://mlsanity.com/paper/2609.39123/) (2026)
- [Holdout Best-of-N: Unbiased Evaluation and Its Cost](https://mlsanity.com/paper/2610.08719/) (2026)

## Abstract (arXiv)

Across science and policy, decision-makers often need to draw conclusions about the best candidate among competing alternatives. For instance, researchers may seek to infer the effectiveness of the most successful treatment or determine which demographic group benefits most from a specific treatment. Similarly, in machine learning, practitioners are often interested in the population performance of the model that performs best empirically. However, cherry-picking the best candidate leads to the winner's curse: the observed performance for the winner is biased upwards, rendering conclusions based on standard measures of uncertainty invalid. We introduce the zoom correction, a novel approach for valid inference on the winner. Our method is flexible: it can be employed in both parametric and nonparametric settings, can handle arbitrary dependencies between candidates, and automatically adapts to the level of selection bias. The method easily extends to important related problems, such as inference on the top k winners, inference on the value and identity of the population winner, and inference on "near-winners."