# An Empirical Study of AI Generated Text Detection Tools

arXiv: https://arxiv.org/abs/2310.01423 · HTML: https://mlsanity.com/paper/2310.01423/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 27 Sep 2023.

## Builds on (most cited first, 15 of 15)

- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion](https://mlsanity.com/paper/2007.02220/) (2020)
- [PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstracts](https://mlsanity.com/paper/1710.06071/) (2017)
- [Testing of Detection Tools for AI-Generated Text](https://mlsanity.com/paper/2306.15666/) (2023)
- [ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text](https://mlsanity.com/paper/2301.13852/) (2023)
- [Artificial Text Detection via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2109.04825/) (2021)
- [Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models](https://mlsanity.com/paper/2209.06506/) (2022)
- [To ChatGPT, or not to ChatGPT: That is the question!](https://mlsanity.com/paper/2304.01487/) (2023)
- [Mining Disinformation and Fake News: Concepts, Methods, and Recent Advancements](https://mlsanity.com/paper/2001.00623/) (2020)
- [Evaluating AIGC Detectors on Code Content](https://mlsanity.com/paper/2304.05193/) (2023)

## Built on by (most cited first, 9 of 9)

- [Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods](https://mlsanity.com/paper/2406.15583/) (2024)
- [Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Review](https://mlsanity.com/paper/2410.03019/) (2024)
- [StyloAI: Distinguishing AI-Generated Content with Stylometric Analysis](https://mlsanity.com/paper/2405.10129/) (2024)
- [Survey on AI-Generated Media Detection: From Non-MLLM to MLLM](https://mlsanity.com/paper/2502.05240/) (2025)
- [Enhancing Text Authenticity: A Novel Hybrid Approach for AI-Generated Text Detection](https://mlsanity.com/paper/2406.06558/) (2024)
- [CodeMirage: A Multi-Lingual Benchmark for Detecting AI-Generated and Paraphrased Source Code from Production-Level LLMs](https://mlsanity.com/paper/2506.11059/) (2025)
- [Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity](https://mlsanity.com/paper/2407.13105/) (2024)
- [Quantitative Analysis of AI-Generated Texts in Academic Research: A Study of AI Presence in Arxiv Submissions using AI Detection Tool](https://mlsanity.com/paper/2403.13812/) (2024)
- [Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals](https://mlsanity.com/paper/2311.14720/) (2023)

## Abstract (arXiv)

Since ChatGPT has emerged as a major AIGC model, providing high-quality responses across a wide range of applications (including software development and maintenance), it has attracted much interest from many individuals. ChatGPT has great promise, but there are serious problems that might arise from its misuse, especially in the realms of education and public safety. Several AIGC detectors are available, and they have all been tested on genuine text. However, more study is needed to see how effective they are for multi-domain ChatGPT material. This study aims to fill this need by creating a multi-domain dataset for testing the state-of-the-art APIs and tools for detecting artificially generated information used by universities and other research institutions. A large dataset consisting of articles, abstracts, stories, news, and product reviews was created for this study. The second step is to use the newly created dataset to put six tools through their paces. Six different artificial intelligence (AI) text identification systems, including "GPTkit," "GPTZero," "Originality," "Sapling," "Writer," and "Zylalab," have accuracy rates between 55.29 and 97.0%. Although all the tools fared well in the evaluations, originality was particularly effective across the board.