# ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literature

arXiv: https://arxiv.org/abs/2403.16887 · HTML: https://mlsanity.com/paper/2403.16887/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2024.

## Built on by (most cited first, 26 of 26)

- [Delving into LLM-assisted writing in biomedical publications through excess vocabulary](https://mlsanity.com/paper/2406.07016/) (2024)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [The Impact of Large Language Models in Academia: from Writing to Speaking](https://mlsanity.com/paper/2409.13686/) (2024)
- [Human-LLM Coevolution: Evidence from Academic Writing](https://mlsanity.com/paper/2502.09606/) (2025)
- [Large Means Left: Political Bias in Large Language Models Increases with Their Number of Parameters](https://mlsanity.com/paper/2505.04393/) (2025)
- [Academic journals' AI policies fail to curb the surge in AI-assisted academic writing](https://mlsanity.com/paper/2512.06705/) (2025)
- [CoDet-M4: Detecting Machine-Generated Code in Multi-Lingual, Multi-Generator and Multi-Domain Settings](https://mlsanity.com/paper/2503.13733/) (2025)
- [System 2 thinking in OpenAI's o1-preview model: Near-perfect performance on a mathematics exam](https://mlsanity.com/paper/2410.07114/) (2024)
- [Are AI Detectors Good Enough? A Survey on Quality of Datasets With Machine-Generated Texts](https://mlsanity.com/paper/2410.14677/) (2024)
- [Why Does ChatGPT "Delve" So Much? Exploring the Sources of Lexical Overrepresentation in Large Language Models](https://mlsanity.com/paper/2412.11385/) (2024)
- [Delving into the Utilisation of ChatGPT in Scientific Publications in Astronomy](https://mlsanity.com/paper/2406.17324/) (2024)
- [Advancing the Scientific Method with Large Language Models: From Hypothesis to Discovery](https://mlsanity.com/paper/2505.16477/) (2025)
- [Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction](https://mlsanity.com/paper/2505.06297/) (2025)
- [Science Is Falling Behind the Frontier: Foundation Model Adoption Across Half a Million Papers](https://mlsanity.com/paper/2511.21739/) (2025)
- [Understanding Generative AI Content with Embedding Models](https://mlsanity.com/paper/2408.10437/) (2024)
- [Model Misalignment and Language Change: Traces of AI-Associated Language in Unscripted Spoken English](https://mlsanity.com/paper/2508.00238/) (2025)
- [NLLG Quarterly arXiv Report 09/24: What are the most influential current AI Papers?](https://mlsanity.com/paper/2412.12121/) (2024)
- [Convergence Dynamics and Stabilization Strategies of Co-Evolving Generative Models](https://mlsanity.com/paper/2503.08117/) (2025)
- [GPT Editors, Not Authors: The Stylistic Footprint of LLMs in Academic Preprints](https://mlsanity.com/paper/2505.17327/) (2025)
- [Word Overuse and Alignment in Large Language Models: The Influence of Learning from Human Feedback](https://mlsanity.com/paper/2508.01930/) (2025)
- [PRAIB: Peer Review AI Benchmark of Behaviour of LLM-Assisted Reviewing](https://mlsanity.com/paper/2605.29815/) (2026)
- [Most biomedical publications show signs of LLM-assisted writing](https://mlsanity.com/paper/2608.10715/) (2026)
- [Detection Avoidance Techniques for Large Language Models](https://mlsanity.com/paper/2503.07595/) (2025)
- [Does Scientific Writing Converge to U.S. English? Evidence from Generative AI-Assisted Publications](https://mlsanity.com/paper/2511.11687/) (2025)
- [Impact of ChatGPT on the writing style of condensed matter physicists](https://mlsanity.com/paper/2408.17325/) (2024)
- [Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learning](https://mlsanity.com/paper/2606.00334/) (2026)

## Abstract (arXiv)

The use of ChatGPT and similar Large Language Model (LLM) tools in scholarly communication and academic publishing has been widely discussed since they became easily accessible to a general audience in late 2022. This study uses keywords known to be disproportionately present in LLM-generated text to provide an overall estimate for the prevalence of LLM-assisted writing in the scholarly literature. For the publishing year 2023, it is found that several of those keywords show a distinctive and disproportionate increase in their prevalence, individually and in combination. It is estimated that at least 60,000 papers (slightly over 1% of all articles) were LLM-assisted, though this number could be extended and refined by analysis of other characteristics of the papers or by identification of further indicative keywords.