mlsanityData to 7 Oct 2026

Is there "Secret Sauce'' in Large Language Model Development?

arXiv 2602.07238 ↗First posted 6 Feb 2026cs.AI cs.LG econ.GNcited by 2

Not on the map. Why?

Abstract (arXiv)

Do leading LLM developers possess a proprietary ``secret sauce'', or is LLM performance driven by scaling up compute? Using training and benchmark data for 809 models released between 2022 and 2025, we estimate scaling-law regressions with release-date and developer fixed effects. We find clear evidence of developer-specific efficiency advantages, but their importance depends on where models lie in the performance distribution. At the frontier, 80-90% of performance differences are explained by higher training compute, implying that scale--not proprietary technology--drives frontier advances. Away from the frontier, however, proprietary techniques and shared algorithmic progress substantially reduce the compute required to reach fixed capability thresholds. Some companies can systematically produce smaller models more efficiently. Strikingly, we also find substantial variation of model efficiency within companies; a firm can train two models with more than 40x compute efficiency difference. We also discuss the implications for AI leadership and capability diffusion.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 8 of its 10 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

8 of 10 references.

References · 8

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  3. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  4. 2024Observational Scaling Laws and the Predictability of Language Model Performancecited by 128
  5. 2024Scaling Laws for Predicting Downstream Performance in LLMscited by 48
  6. 2024Algorithmic progress in language modelscited by 31
  7. 2021Scaling Laws for Deep Learningcited by 25
  8. 2025Rethinking LLM Advancement: Compute-Dependent and Independent Paths to Progresscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.