mlsanityData to 7 Oct 2026

Large Language Monkeys: Scaling Inference Compute with Repeated Sampling

arXiv 2407.21787 ↗First posted 31 Jul 2024cs.LG cs.AIcited by 946

Not on the map. Why?

Cited by 549 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit models to making only one attempt at a problem. Here, we explore inference compute as another axis for scaling, using the simple technique of repeatedly sampling candidate solutions from a model. Across multiple tasks and models, we observe that coverage -- the fraction of problems that are solved by any generated sample -- scales with the number of samples over four orders of magnitude. Interestingly, the relationship between coverage and the number of samples is often log-linear and can be modelled with an exponentiated power law, suggesting the existence of inference-time scaling laws. In domains like coding and formal proofs, where answers can be automatically verified, these increases in coverage directly translate into improved performance. When we apply repeated sampling to SWE-bench Lite, the fraction of issues solved with DeepSeek-Coder-V2-Instruct increases from 15.9% with one sample to 56% with 250 samples, outperforming the single-sample state-of-the-art of 43%. In domains without automatic verifiers, we find that common methods for picking from a sample collection (majority voting and reward models) plateau beyond several hundred samples and fail to fully scale with the sample budget.

Key facts. Cited by 946 papers in the map (data through 7 Oct 2026). Builds on 42 of its 58 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

42 of 58 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  12. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
30 more
  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  2. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  3. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  5. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  6. 2021Measuring Coding Challenge Competence With APPScited by 879
  7. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  8. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  9. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  10. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  11. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  12. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  13. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 420
  14. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  15. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 369
  16. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  17. 2019SPoC: Search-based Pseudocode to Codecited by 176
  18. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 123
  19. 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 113
  20. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 110
  21. 2024Knowledge Fusion of Large Language Modelscited by 110
  22. 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 71
  23. 2021Scaling Scaling Laws with Board Gamescited by 62
  24. 2023Rewarding Chatbots for Real-World Engagement with Millions of Userscited by 58
  25. 2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 54
  26. 2024The Larger the Better? Improved LLM Code-Generation via Budget Reallocationcited by 41
  27. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Timecited by 40
  28. 2024Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Designcited by 13
  29. 2024When is the consistent prediction likely to be a correct prediction?cited by 8
  30. 2024Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 87; 2025: 452; 2026: 407.

946 papers.

Sort:

By area:

  1. 2025s1: Simple test-time scalingcited by 1,311
  2. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  3. 2025LIMO: Less is More for Reasoningcited by 488
  4. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  5. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  6. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  7. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  8. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  9. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  10. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
  11. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
  12. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
38 more
  1. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 225
  2. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  3. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
  4. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  5. 2025Deep Think with Confidencecited by 199
  6. 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
  7. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 194
  8. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
  9. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  10. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  11. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  12. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  13. 2024Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agentscited by 144
  14. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 137
  15. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalingcited by 135
  16. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 134
  17. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  18. 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinementcited by 124
  19. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoningcited by 118
  20. 2025Process Reward Models That Thinkcited by 118
  21. 2024CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmarkcited by 115
  22. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 108
  23. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  24. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoningcited by 106
  25. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 102
  26. 2025S*: Test Time Scaling for Code Generationcited by 95
  27. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approachcited by 95
  28. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 94
  29. 2024Scaling Laws for Precisioncited by 92
  30. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 91
  31. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 90
  32. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 88
  33. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 88
  34. 2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Modelscited by 86
  35. 2024VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Modelscited by 85
  36. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scalingcited by 77
  37. 2025Efficient Test-Time Scaling via Self-Calibrationcited by 74
  38. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Timecited by 73

50 most cited of 946 All 946 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  2. 2025OptScale: Probabilistic Optimality for Inference-time Scaling
  3. 2025The Art of Scaling Test-Time Compute for Large Language Models
  4. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
  5. 2024Observational Scaling Laws and the Predictability of Language Model Performance
  6. 2026Ladders in Chaos: When, How, (and Perhaps Why) Does Test-Time Scaling Improve LLM Machine Translation
  7. 2025On the Role of Temperature Sampling in Test-Time Scaling
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters