# Is ChatGPT Transforming Academics' Writing Style?

arXiv: https://arxiv.org/abs/2404.08627 · HTML: https://mlsanity.com/paper/2404.08627/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Builds on (most cited first, 7 of 7)

- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://mlsanity.com/paper/2403.07183/) (2024)
- [Mapping the Increasing Use of LLMs in Scientific Papers](https://mlsanity.com/paper/2404.01268/) (2024)
- [The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4](https://mlsanity.com/paper/2311.07361/) (2023)
- [Extracting Accurate Materials Data from Research Papers with Conversational Language Models and Prompt Engineering](https://mlsanity.com/paper/2303.05352/) (2023)
- [ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing](https://mlsanity.com/paper/2303.13367/) (2023)

## Built on by (most cited first, 26 of 26)

- [Delving into LLM-assisted writing in biomedical publications through excess vocabulary](https://mlsanity.com/paper/2406.07016/) (2024)
- [Benchmarking Linguistic Diversity of Large Language Models](https://mlsanity.com/paper/2412.10271/) (2024)
- [Empirical evidence of Large Language Model's influence on human spoken communication](https://mlsanity.com/paper/2409.01754/) (2024)
- [The Impact of Large Language Models in Academia: from Writing to Speaking](https://mlsanity.com/paper/2409.13686/) (2024)
- [AIGS: Generating Science from AI-Powered Automated Falsification](https://mlsanity.com/paper/2411.11910/) (2024)
- [SilverSpeak: Evading AI-Generated Text Detectors using Homoglyphs](https://mlsanity.com/paper/2406.11239/) (2024)
- [Delving into the Utilisation of ChatGPT in Scientific Publications in Astronomy](https://mlsanity.com/paper/2406.17324/) (2024)
- [Examining Linguistic Shifts in Academic Writing Before and After the Launch of ChatGPT: A Study on Preprint Papers](https://mlsanity.com/paper/2505.12218/) (2025)
- [How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses](https://mlsanity.com/paper/2504.13038/) (2025)
- [Understanding Generative AI Content with Embedding Models](https://mlsanity.com/paper/2408.10437/) (2024)
- [What is the Role of Large Language Models in the Evolution of Astronomy Research?](https://mlsanity.com/paper/2409.20252/) (2024)
- [Wikipedia in the Era of LLMs: Evolution and Risks](https://mlsanity.com/paper/2503.02879/) (2025)
- [NLLG Quarterly arXiv Report 09/24: What are the most influential current AI Papers?](https://mlsanity.com/paper/2412.12121/) (2024)
- [On the Detectability of LLM-Generated Text: What Exactly Is LLM-Generated Text?](https://mlsanity.com/paper/2510.20810/) (2025)
- [Generative AI Tools in Academic Research: Applications and Implications for Qualitative and Quantitative Research Methodologies](https://mlsanity.com/paper/2408.06872/) (2024)
- [Humans Coexist, So Must Embodied Artificial Agents](https://mlsanity.com/paper/2502.04809/) (2025)
- [Most biomedical publications show signs of LLM-assisted writing](https://mlsanity.com/paper/2608.10715/) (2026)
- [Exploring the change in scientific readability following the release of ChatGPT](https://mlsanity.com/paper/2506.21825/) (2025)
- [Does Scientific Writing Converge to U.S. English? Evidence from Generative AI-Assisted Publications](https://mlsanity.com/paper/2511.11687/) (2025)
- [Accepted with Minor Revisions: Value of AI-Assisted Scientific Writing](https://mlsanity.com/paper/2511.12529/) (2025)
- [Beyond Via: Analysis and Estimation of the Impact of Large Language Models in Academic Papers](https://mlsanity.com/paper/2603.25638/) (2026)
- [Impact of large language models on peer review opinions from a fine-grained perspective: Evidence from top conference proceedings in AI](https://mlsanity.com/paper/2604.19578/) (2026)
- [Impact of ChatGPT on the writing style of condensed matter physicists](https://mlsanity.com/paper/2408.17325/) (2024)
- [Co-Creative Learning via Metropolis-Hastings Interaction between Humans and AI](https://mlsanity.com/paper/2506.15468/) (2025)
- [Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learning](https://mlsanity.com/paper/2606.00334/) (2026)
- [War in the Abstract: The Rise and Consequences of Militarized Language in Scientific Communication](https://mlsanity.com/paper/2606.23462/) (2026)

## Abstract (arXiv)

Based on one million arXiv papers submitted from May 2018 to January 2024, we assess the textual density of ChatGPT's writing style in their abstracts through a statistical analysis of word frequency changes. Our model is calibrated and validated on a mixture of real abstracts and ChatGPT-modified abstracts (simulated data) after a careful noise analysis. The words used for estimation are not fixed but adaptive, including those with decreasing frequency. We find that large language models (LLMs), represented by ChatGPT, are having an increasing impact on arXiv abstracts, especially in the field of computer science, where the fraction of LLM-style abstracts is estimated to be approximately 35%, if we take the responses of GPT-3.5 to one simple prompt, "revise the following sentences", as a baseline. We conclude with an analysis of both positive and negative aspects of the penetration of LLMs into academics' writing style.