mlsanityData to 7 Oct 2026

Science Is Falling Behind the Frontier: Foundation Model Adoption Across Half a Million Papers

arXiv 2511.21739 ↗First posted 21 Nov 2025cs.DL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

We present the first large-scale analysis of AI foundation model usage in science -- not just citations or keywords. We find that adoption has grown rapidly, at nearly-exponential rates, with the highest uptake in Linguistics, Computer Science, and Engineering. Vision models are the most used foundation models in science, although language models' share is growing. Open-weight models dominate. As AI builders increase the parameter counts of their models, scientists have followed suit but at a much slower rate: in 2015, the mean foundation model adopted in science was 5.4x larger than the mean model being built; by 2024 that relationship had reversed, with the mean model built 6.9x larger than the mean model adopted. We also present suggestive evidence that scientists' use of these smaller models may be limiting them from getting the full benefits of AI-enabled science, as papers that use larger models appear in higher-impact journals and accrue more citations.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 36 of its 75 references that are in the map. First posted on arXiv 21 Nov 2025.

Builds on

36 of 75 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 240
  10. 2019A Constructive Prediction of the Generalization Error Across Scalescited by 202
  11. 2022Can Foundation Models Wrangle Your Data?cited by 173
  12. 2022Compute Trends Across Three Eras of Machine Learningcited by 161
24 more
  1. 2019Structural Scaffolds for Citation Intent Classification in Scientific Publicationscited by 129
  2. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 128
  3. 2018Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysiscited by 110
  4. 2023The Semantic Scholar Open Data Platformcited by 109
  5. 2020The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Researchcited by 67
  6. 2024On the Societal Impact of Open Foundation Modelscited by 64
  7. 2023The Foundation Model Transparency Indexcited by 63
  8. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  9. 2024OmniJet-α: The first cross-task foundation model for particle physicscited by 41
  10. 2022Machine Learning Model Sizes and the Parameter Gapcited by 39
  11. 2021MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label settingcited by 30
  12. 2024ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literaturecited by 25
  13. 2024The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?cited by 20
  14. 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Modelscited by 14
  15. 2023CitePrompt: Using Prompts to Identify Citation Intent in Scientific Paperscited by 12
  16. 2023The Future of Fundamental Science Led by Generative Closed-Loop Artificial Intelligencecited by 12
  17. 2025Physics-Guided Foundation Model for Scientific Discovery: An Application to Aquatic Sciencecited by 12
  18. 2022VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understandingcited by 11
  19. 2023Artificial intelligence adoption in the physical sciences, natural sciences, life sciences, social sciences and the arts and humanities: A bibliometric analysis of research publications from 1960-2021cited by 8
  20. 2024Oil & Water? Diffusion of AI Within and Across Scientific Fieldscited by 6
  21. 2022Foundation models in brief: A historical, socio-technical focuscited by 3
  22. 2024Reducing the Barriers to Entry for Foundation Model Trainingcited by 3
  23. 2023Quantifying the Benefit of Artificial Intelligence for Scientific Researchcited by 2
  24. 2025The Role of Computing Resources in Publishing Foundation Model Researchcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026The Shrinking Lifespan of LLMs in Sciencecited by 1
  2. 2026When to Trust, How to Distill: Multi-Foundation Model Guidance for Lightweight, Robust Scientific Time Series Forecastingcited by 1
  3. 2026Automating Detection and Root-Cause Analysis of Flaky Tests in Quantum Softwarecited by 0
  4. 2026Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teamscited by 0
  5. 2026AI in Science: Early Insightscited by 0