# When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scaling

arXiv: https://arxiv.org/abs/2606.28661 · HTML: https://mlsanity.com/paper/2606.28661/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 27 Jun 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 25 of 25)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models](https://mlsanity.com/paper/2408.00724/) (2024)
- [A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?](https://mlsanity.com/paper/2503.24235/) (2025)
- [Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling](https://mlsanity.com/paper/2502.06703/) (2025)
- [Great Models Think Alike and this Undermines AI Oversight](https://mlsanity.com/paper/2502.04313/) (2025)
- [How Do Large Language Monkeys Get Their Power (Laws)?](https://mlsanity.com/paper/2502.17578/) (2025)
- [A Simple Model of Inference Scaling Laws](https://mlsanity.com/paper/2410.16377/) (2024)
- [Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels](https://mlsanity.com/paper/2605.29800/) (2026)
- [Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems](https://mlsanity.com/paper/2403.02419/) (2024)
- [On the Effect of Sampling Diversity in Scaling LLM Inference](https://mlsanity.com/paper/2502.11027/) (2025)
- [Efficient Prediction of Pass@k Scaling in Large Language Models](https://mlsanity.com/paper/2510.05197/) (2025)
- [Machine Learning vs Deep Learning: The Generalization Problem](https://mlsanity.com/paper/2403.01621/) (2024)
- [Model Capability Dominates: Inference-Time Optimization Lessons from AIMO 3](https://mlsanity.com/paper/2603.27844/) (2026)
- [Solve for the Hyperparameter, Skip the Search: Kolmogorov-Optimal Scaling Laws for Spline Regression](https://mlsanity.com/paper/2606.23575/) (2026)
- [No 3D Matrices: A Unified Tensor-Product View of Matrix-Free Cartesian PDE Solvers](https://mlsanity.com/paper/2606.25148/) (2026)

## Built on by (most cited first, 6 of 6)

- [GRPO, Dr. GRPO, and DAPO Are Three Operations on One Number: The Group-Standard-Deviation Identity](https://mlsanity.com/paper/2607.00152/) (2026)
- [SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning](https://mlsanity.com/paper/2607.26873/) (2026)
- [RLVR is a Kernel, Not a Function: Statistical Inference for pass@k Crossovers](https://mlsanity.com/paper/2609.22547/) (2026)
- [The Exceedance Design Effect: Effective Sample Size for Thresholds under Clustering](https://mlsanity.com/paper/2608.21262/) (2026)
- [Self-Repulsive Sampling for Diffusion Language Models](https://mlsanity.com/paper/2609.39560/) (2026)
- [From Discovery to Decision: Finite-Budget Recoverability in LLM Voting](https://mlsanity.com/paper/2610.01014/) (2026)

## Abstract (arXiv)

People overthink; language models over-sample, and the extra effort can talk both into a worse answer. Reasoning systems answer a hard question by sampling it many times (test-time scaling), and the more they draw, the more often a correct answer turns up somewhere, so coverage, the fraction of problems with at least one correct try, climbs and appears to be progress. But a deployed system must return one answer, and choosing it, not knowing which try is right, is selection; selection is capped, and past a point extra samples only make the model surer of a confident mistake, even as every draw adds cost. The gap between climbing coverage and stalled selection, the identifiability gap, is the answer a model can produce but not pick. So the real question is not whether to sample but how far, and the answer is: not far. For picking an answer, the vote has already settled within a few dozen draws, the modal ceiling; for scoring a benchmark, sooner still, the correlation ceiling. Beyond that, extra draws cost compute and add nothing, and can even make the answer worse. This paper turns the cutoff into a single number, the effective number of samples, that any sampling run already reveals. The bottleneck is recognizing a right answer, not generating one.