mlsanityData to 7 Oct 2026

Prescriptive Scaling Reveals the Evolution of Language Model Capabilities

arXiv 2602.15327 ↗First posted 17 Feb 2026cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Machine learning model performance improvements tend to arise from competition and application. For deployment, we consider prescriptive scaling laws: given a pre-training compute budget, what downstream accuracy is attainable with contemporary post-training practice, and how stable is that mapping as the field evolves? Using large-scale observational evaluations with 5k existing and 2k newly evaluated model checkpoints spanning 2022-2026 across six benchmarks, we estimate capability boundaries, high conditional quantiles of benchmark scores as a function of log pre-training FLOPs, via smoothed quantile regression with a monotone, saturating sigmoid parameterization. We validate temporal reliability by fitting on earlier model generations and evaluating on later releases: across four of six tasks, the out-of-distribution coverage error remains below 2%, while math reasoning exhibits a consistently advancing boundary over time. For instance, at a budget of 10^24 FLOPs, the estimated attainable accuracies are 0.83 on IFEval and 0.54 on MATH Lvl 5. We then extend our approach to analyze task-dependent saturation and to probe contamination-related shifts on math reasoning tasks. Finally, we introduce a balanced I-optimal sampling algorithm that recovers near-full-data frontiers using roughly 20% of the parameter-count-weighted evaluation budget, as low as 5% on some tasks, while maintaining comparable calibration. Together, our work releases Proteus-2k, the latest model performance evaluation dataset, and introduces a practical methodology for translating compute budgets into reliable performance expectations and for monitoring when capability boundaries shift across time.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 48 of its 63 references that are in the map. First posted on arXiv 17 Feb 2026.

Builds on

48 of 63 references.

References · 48

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2024Qwen2.5 Technical Reportcited by 4,713
  10. 2023Mistral 7Bcited by 3,065
  11. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
36 more
  1. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  2. 2024Qwen2 Technical Reportcited by 2,076
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  5. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  6. 2025Gemma 3 Technical Reportcited by 1,686
  7. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  8. 2024Yi: Open Foundation Models by 01.AIcited by 706
  9. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  10. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  11. 2018An Empirical Model of Large-Batch Trainingcited by 268
  12. 2021Scaling Laws for Transfercited by 265
  13. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
  14. 2025Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)cited by 137
  15. 2024Observational Scaling Laws and the Predictability of Language Model Performancecited by 128
  16. 2024Language models scale reliably with over-training and on downstream taskscited by 105
  17. 2022Broken Neural Scaling Lawscited by 102
  18. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  19. 2025Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 85
  20. 2022Mapping global dynamics of benchmark creation and saturation in artificial intelligencecited by 71
  21. 2024How Does Critical Batch Size Scale in Pre-training?cited by 68
  22. 2024Scaling Laws for Predicting Downstream Performance in LLMscited by 48
  23. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 45
  24. 2024Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?cited by 39
  25. 2024Selecting Large Language Model to Fine-tune via Rectified Scaling Lawcited by 33
  26. 2024Training on the Test Task Confounds Evaluation and Emergencecited by 33
  27. 2025Scaling Laws Are Unreliable for Downstream Tasks: A Reality Checkcited by 27
  28. 2025How Benchmark Prediction from Fewer Data Misses the Markcited by 26
  29. 2024CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-trainingcited by 20
  30. 2025Language Models Improve When Pretraining Data Matches Target Taskscited by 18
  31. 2025EvoLM: In Search of Lost Language Model Training Dynamicscited by 16
  32. 2025Knowledge or Reasoning? A Close Look at How LLMs Think Across Domainscited by 14
  33. 2025Train-before-Test Harmonizes Language Model Rankingscited by 12
  34. 2025Unveiling Downstream Performance Scaling of LLMs: A Clustering-Based Perspectivecited by 7
  35. 2025Revisiting the Scaling Properties of Downstream Metrics in Large Language Model Trainingcited by 6
  36. 2025Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learningcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
  2. 2024Revisiting the Superficial Alignment Hypothesis
  3. 2025Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning
  4. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead
  5. 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
  6. 2026What do Language Models Learn and When? The Implicit Curriculum Hypothesis
  7. 2026Brevity Constraints Reverse Performance Hierarchies in Language Models
  8. 2026Operationalising the Superficial Alignment Hypothesis via Task Complexity