# Detecting LLM-assisted writing in scientific communication: Are we there yet?

arXiv: https://arxiv.org/abs/2401.16807 · HTML: https://mlsanity.com/paper/2401.16807/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 18 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Builds on (most cited first, 6 of 6)

- [DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2306.05540/) (2023)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [LLMDet: A Third Party Large Language Models Generated Text Detection Tool](https://mlsanity.com/paper/2305.15004/) (2023)
- [ConDA: Contrastive Domain Adaptation for AI-generated Text Detection](https://mlsanity.com/paper/2309.03992/) (2023)
- [A Computational Model For Individual Scholars' Writing Style Dynamics](https://mlsanity.com/paper/2305.04900/) (2023)
- [The Scientometrics and Reciprocality Underlying Co-Authorship Panels in Google Scholar Profiles](https://mlsanity.com/paper/2308.07001/) (2023)

## Built on by (most cited first, 4 of 4)

- [Delving into LLM-assisted writing in biomedical publications through excess vocabulary](https://mlsanity.com/paper/2406.07016/) (2024)
- [Investigating Tax Evasion Emergence Using Dual Large Language Model and Deep Reinforcement Learning Powered Agent-based Simulation](https://mlsanity.com/paper/2501.18177/) (2025)
- [Most biomedical publications show signs of LLM-assisted writing](https://mlsanity.com/paper/2608.10715/) (2026)
- [Detoxify: A framework for abusive text transformation using LLMs](https://mlsanity.com/paper/2507.10177/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs), exemplified by ChatGPT, have significantly reshaped text generation, particularly in the realm of writing assistance. While ethical considerations underscore the importance of transparently acknowledging LLM use, especially in scientific communication, genuine acknowledgment remains infrequent. A potential avenue to encourage accurate acknowledging of LLM-assisted writing involves employing automated detectors. Our evaluation of four cutting-edge LLM-generated text detectors reveals their suboptimal performance compared to a simple ad-hoc detector designed to identify abrupt writing style changes around the time of LLM proliferation. We contend that the development of specialized detectors exclusively dedicated to LLM-assisted writing detection is necessary. Such detectors could play a crucial role in fostering more authentic recognition of LLM involvement in scientific communication, addressing the current challenges in acknowledgment practices.