mlsanityData to 14 Oct 2026

The Limits of Inference Scaling Through Resampling

arXiv 2411.17501 ↗First posted 26 Nov 2024cs.LG cs.AIcited by 53

Not on the map. Why?

Cited by 27 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent research has generated hope that inference scaling, such as resampling solutions until they pass verifiers like unit tests, could allow weaker models to match stronger ones. Beyond inference, this approach also enables training reasoning models, where data is curated using rejection sampling against a verifier. However, we show that this approach is fundamentally limited when verifiers are imperfect and have a non-zero probability of producing false positives. Resampling cannot decrease this probability, so it imposes an upper bound to the accuracy of resampling-based inference scaling, regardless of compute budget. Our analysis shows that there is a strong correlation between the model's single-sample accuracy and its false positive rate on HumanEval and MBPP, whose unit tests have limited coverage. Therefore, no amount of inference scaling of weaker models can enable them to match the single-sample accuracy of a sufficiently strong model. Empirical results show that optimal sampling attempts are often fewer than 10, as the negative utility of false positives outweighs benefits, bending inference scaling curves downward. Finally, false positives may have other undesirable qualities, like poor adherence to coding style conventions.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 62 of its 70 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

62 of 70 references.

References · 62

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  11. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  12. 2016Concrete Problems in AI Safetycited by 1,642
38 more
  1. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  3. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
  4. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  5. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  6. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 480
  7. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
  8. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 427
  9. 2023Llemma: An Open Language Model For Mathematicscited by 388
  10. 2022CodeT: Code Generation with Generated Testscited by 384
  11. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  12. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 307
  13. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  14. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
  15. 2024DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Datacited by 234
  16. 2024AI Agents That Mattercited by 219
  17. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
  18. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 214
  19. 2024More Agents Is All You Needcited by 180
  20. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 179
  21. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 137
  22. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
  23. 2023Baldur: Whole-Proof Generation and Repair with Large Language Modelscited by 119
  24. 2020Unit Test Case Generation with Transformers and Focal Contextcited by 118
  25. 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 116
  26. 2023LEGO-Prover: Neural Theorem Proving with Growing Librariescited by 109
  27. 2024Common 7B Language Models Already Possess Strong Math Capabilitiescited by 109
  28. 2023ChatUniTest: A Framework for LLM-Based Test Generationcited by 96
  29. 2024Prover-Verifier Games improve legibility of LLM outputscited by 85
  30. 2024Planning In Natural Language Improves LLM Search For Code Generationcited by 83
  31. 2023An In-Context Learning Agent for Formal Theorem-Provingcited by 82
  32. 2023TinyGSM: achieving >80% on GSM8k with small language modelscited by 70
  33. 2024TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generationcited by 68
  34. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 60
  35. 2023Using Large Language Models to Generate JUnit Tests: An Empirical Studycited by 53
  36. 2022Pynguin: Automated Unit Test Generation for Pythoncited by 50
  37. 2023Improving Large Language Model Fine-tuning for Solving Math Problemscited by 49
  38. 2024When is Tree Search Useful for LLM Planning? It Depends on the Discriminatorcited by 47

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 32; 2026: 19.

53 papers.

Sort:

By area:

  1. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  2. 2025Establishing Best Practices for Building Rigorous Agentic Benchmarkscited by 102
  3. 2025S*: Test Time Scaling for Code Generationcited by 98
  4. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 97
  5. 2024Best-of-N Jailbreakingcited by 76
  6. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 50
  7. 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
  8. 2025From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?cited by 23
  9. 2026Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokenscited by 22
  10. 2025Incentivizing LLMs to Self-Verify Their Answerscited by 17
  11. 2025AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Taskscited by 16
  12. 2025PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Spacecited by 16
38 more
  1. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  2. 2025Generating Symbolic World Models via Test-time Scaling of Large Language Modelscited by 12
  3. 2025Multi2: Multi-Agent Test-Time Scalable Framework for Multi-Document Processingcited by 12
  4. 2025Better Process Supervision with Bi-directional Rewarding Signalscited by 12
  5. 2025Examining False Positives under Inference Scaling for Mathematical Reasoningcited by 11
  6. 2025VerifiAgent: a Unified Verification Agent in Language Model Reasoningcited by 10
  7. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoningcited by 9
  8. 2025When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMscited by 9
  9. 2025Trust but Verify! A Survey on Verification Design for Test-time Scalingcited by 8
  10. 2025Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflowscited by 7
  11. 2026Scaling Agentic Verifier for Competitive Codingcited by 7
  12. 2025Diverse Inference and Verification for Advanced Reasoningcited by 6
  13. 2025Making, not Taking, the Best of Ncited by 6
  14. 2025Toward a Safe Internet of Agentscited by 6
  15. 2025Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoningcited by 6
  16. 2025On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inferencecited by 6
  17. 2026When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVRcited by 4
  18. 2026Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Failcited by 4
  19. 2025Using Reasoning Models to Generate Search Heuristics that Solve Open Instances of Combinatorial Design Problemscited by 3
  20. 2025Calibrated Predictive Lower Bounds on Time-to-Unsafe-Sampling in LLMscited by 3
  21. 2026Revisiting the (Sub)Optimality of Best-of-N for Inference-Time Alignmentcited by 3
  22. 2025LangProBe: a Language Programs Benchmarkcited by 2
  23. 2025Beyond Theorem Proving: Formulation, Framework and Benchmark for Formal Problem-Solvingcited by 2
  24. 2026Quantifying Empirical Compute-Supervision Tradeoffs in RLVRcited by 2
  25. 2026Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaborationcited by 2
  26. 2026Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agentscited by 2
  27. 2025Beyond Release: Access Considerations for Generative AI Systemscited by 1
  28. 2025Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factualitycited by 1
  29. 2025From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMscited by 1
  30. 2026Fine-Tuning Small Reasoning Models for Quantum Field Theorycited by 1
  31. 2026Resample or Reroute? Recoverable Stopping Debt Without Identified Action Selectioncited by 1
  32. 2026Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluationcited by 1
  33. 2025Instance-Wise Adaptive Sampling for Dataset Construction in Approximating Inverse Problem Solutionscited by 0
  34. 2026The Persistent Vulnerability of Aligned AI Systemscited by 0
  35. 2026ADE: Agentic Data Evolution Framework for Human-Centered Objectivescited by 0
  36. 2026Knowing Is Not Choosing: What Explicit Verification Adds Beyond Generative Preferencecited by 0
  37. 2026From Dead Code and Static Requirements to Working Engines: Software Revival with Coding Agentscited by 0
  38. 2026Cheap to Draw, Expensive to Trust: Certifying Test-Time Scaling Curvescited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics
  2. 2024Keep Guessing? When Considering Inference Scaling, Mind the Baselines
  3. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
  4. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  5. 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification
  6. 2025On the Effect of Sampling Diversity in Scaling LLM Inference
  7. 2025OptScale: Probabilistic Optimality for Inference-time Scaling
  8. 2026Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation