# Science Is Falling Behind the Frontier: Foundation Model Adoption Across Half a Million Papers

arXiv: https://arxiv.org/abs/2511.21739 · HTML: https://mlsanity.com/paper/2511.21739/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 75 references that are in the map.
- First posted on arXiv 21 Nov 2025.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Nougat: Neural Optical Understanding for Academic Documents](https://mlsanity.com/paper/2308.13418/) (2023)
- [A Constructive Prediction of the Generalization Error Across Scales](https://mlsanity.com/paper/1909.12673/) (2019)
- [Can Foundation Models Wrangle Your Data?](https://mlsanity.com/paper/2205.09911/) (2022)
- [Compute Trends Across Three Eras of Machine Learning](https://mlsanity.com/paper/2202.05924/) (2022)
- [Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior](https://mlsanity.com/paper/2306.00258/) (2023)
- [Structural Scaffolds for Citation Intent Classification in Scientific Publications](https://mlsanity.com/paper/1904.01608/) (2019)
- [The Semantic Scholar Open Data Platform](https://mlsanity.com/paper/2301.10140/) (2023)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research](https://mlsanity.com/paper/2010.15581/) (2020)
- [The Foundation Model Transparency Index](https://mlsanity.com/paper/2310.12941/) (2023)
- [On the Societal Impact of Open Foundation Models](https://mlsanity.com/paper/2403.07918/) (2024)
- [Large AI Models in Health Informatics: Applications, Challenges, and the Future](https://mlsanity.com/paper/2303.11568/) (2023)
- [OmniJet-α: The first cross-task foundation model for particle physics](https://mlsanity.com/paper/2403.05618/) (2024)
- [Machine Learning Model Sizes and the Parameter Gap](https://mlsanity.com/paper/2207.02852/) (2022)
- [MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label setting](https://mlsanity.com/paper/2107.00414/) (2021)
- [ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literature](https://mlsanity.com/paper/2403.16887/) (2024)
- [The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?](https://mlsanity.com/paper/2401.02452/) (2024)
- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding](https://mlsanity.com/paper/2211.00430/) (2022)
- [CitePrompt: Using Prompts to Identify Citation Intent in Scientific Papers](https://mlsanity.com/paper/2304.12730/) (2023)
- [The Future of Fundamental Science Led by Generative Closed-Loop Artificial Intelligence](https://mlsanity.com/paper/2307.07522/) (2023)
- [Physics-Guided Foundation Model for Scientific Discovery: An Application to Aquatic Science](https://mlsanity.com/paper/2502.06084/) (2025)

## Built on by (most cited first, 5 of 5)

- [The Shrinking Lifespan of LLMs in Science](https://mlsanity.com/paper/2604.07530/) (2026)
- [When to Trust, How to Distill: Multi-Foundation Model Guidance for Lightweight, Robust Scientific Time Series Forecasting](https://mlsanity.com/paper/2606.19363/) (2026)
- [Automating Detection and Root-Cause Analysis of Flaky Tests in Quantum Software](https://mlsanity.com/paper/2603.09029/) (2026)
- [Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams](https://mlsanity.com/paper/2605.27404/) (2026)
- [AI in Science: Early Insights](https://mlsanity.com/paper/2609.28504/) (2026)

## Abstract (arXiv)

We present the first large-scale analysis of AI foundation model usage in science -- not just citations or keywords. We find that adoption has grown rapidly, at nearly-exponential rates, with the highest uptake in Linguistics, Computer Science, and Engineering. Vision models are the most used foundation models in science, although language models' share is growing. Open-weight models dominate. As AI builders increase the parameter counts of their models, scientists have followed suit but at a much slower rate: in 2015, the mean foundation model adopted in science was 5.4x larger than the mean model being built; by 2024 that relationship had reversed, with the mean model built 6.9x larger than the mean model adopted. We also present suggestive evidence that scientists' use of these smaller models may be limiting them from getting the full benefits of AI-enabled science, as papers that use larger models appear in higher-impact journals and accrue more citations.