mlsanityData to 14 Oct 2026

A Simple Model of Inference Scaling Laws

arXiv 2410.16377 ↗First posted 21 Oct 2024stat.ML cs.AI cs.ITcited by 36

Not on the map. Why?

Cited by 21 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural scaling laws have garnered significant interest due to their ability to predict model performance as a function of increasing parameters, data, and compute. In this work, we propose a simple statistical ansatz based on memorization to study scaling laws in the context of inference, specifically how performance improves with multiple inference attempts. We explore the coverage, or pass@k metric, which measures the chance of success over repeated attempts and provide a motivation for the observed functional form of the inference scaling behavior of the coverage in large language models (LLMs) on reasoning tasks. We then define an "inference loss", which exhibits a power law decay as the number of trials increases, and connect this result with prompting costs. We further test our construction by conducting experiments on a simple generative model, and find that our predictions are in agreement with the empirical coverage curves in a controlled setting. Our simple framework sets the ground for incorporating inference scaling with other known scaling laws.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 18 of its 23 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

18 of 23 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  5. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  6. 2017Deep Learning Scaling is Predictable, Empiricallycited by 582
  7. 2020Scaling Laws for Autoregressive Generative Modelingcited by 522
  8. 2019A Constructive Prediction of the Generalization Error Across Scalescited by 210
  9. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  10. 2022A Solvable Model of Neural Scaling Lawscited by 116
  11. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 114
  12. 2021The Efficiency Misnomercited by 98
6 more
  1. 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 96
  2. 2021Learning Curve Theorycited by 85
  3. 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 73
  4. 2024Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Samplingcited by 61
  5. 2023Scaling Laws for Associative Memoriescited by 35
  6. 2018Critical Percolation as a Framework to Analyze the Training of Deep Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 17.

36 papers.

Sort:

By area:

  1. 2025s1: Simple test-time scalingcited by 1,341
  2. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?cited by 58
  3. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
  4. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  5. 2025Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoningcited by 29
  6. 2025To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoningcited by 24
  7. 2025Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectivenesscited by 20
  8. 2025Efficient Prediction of Pass@k Scaling in Large Language Modelscited by 16
  9. 2025Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Onescited by 13
  10. 2026AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabelingcited by 13
  11. 2025PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chainscited by 10
  12. 2025First Finish Search: Efficient Test-Time Scaling in Large Language Modelscited by 9
24 more
  1. 2025CyberV: Cybernetics for Test-time Scaling in Video Understandingcited by 8
  2. 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scalingcited by 8
  3. 2025Predictable Scale: Part II, Farseer: A Refined Scaling Law in Large Language Modelscited by 6
  4. 2026Learning Shrinks the Hard Tail: Training-Dependent Inference Scaling in a Solvable Linear Modelcited by 6
  5. 2026When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scalingcited by 6
  6. 2025A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Searchcited by 5
  7. 2025Pretraining Scaling Laws for Generative Evaluations of Language Modelscited by 5
  8. 2025Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scalingcited by 4
  9. 2026MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barriercited by 4
  10. 2026Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossovercited by 4
  11. 2025X-Factor: Quality Is a Dataset-Intrinsic Propertycited by 3
  12. 2025Evaluating the Robustness of Chinchilla Compute-Optimal Scalingcited by 2
  13. 2026Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decodingcited by 2
  14. 2026More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)cited by 2
  15. 2026Predicting and improving test-time scaling laws via reward tail-guided searchcited by 2
  16. 2026Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimationcited by 2
  17. 2025Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factualitycited by 1
  18. 2026Hard or Just Unreached? Diagnosing the Sampling Blind Spot in Math-Reasoning Difficulty Estimationcited by 1
  19. 2026Continually self-improving AIcited by 0
  20. 2026CoTEvol: Self-Evolving Chain-of-Thoughts for Data Synthesis in Mathematical Reasoningcited by 0
  21. 2026Sketch-and-Verify: Structured Inference-Time Scaling via Program Sketchingcited by 0
  22. 2026An Asymptotic Theory of Chain-of-Thought in In-Context Learningcited by 0
  23. 2026The Inference-Compute Frontier and a Latency-Efficient Architecture for Limit Order Book Predictioncited by 0
  24. 2026Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreakingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  2. 2024Keep Guessing? When Considering Inference Scaling, Mind the Baselines
  3. 2025Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals
  4. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead
  5. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
  6. 2025The Art of Scaling Test-Time Compute for Large Language Models
  7. 2025On the Role of Temperature Sampling in Test-Time Scaling
  8. 2025Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models