# Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT

arXiv: https://arxiv.org/abs/2308.05341 · HTML: https://mlsanity.com/paper/2308.05341/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reports](https://mlsanity.com/paper/2212.14882/) (2022)
- [ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text](https://mlsanity.com/paper/2301.13852/) (2023)
- [Stylometric Detection of AI-Generated Text in Twitter Timelines](https://mlsanity.com/paper/2303.03697/) (2023)
- [Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis](https://mlsanity.com/paper/2304.05534/) (2023)
- [Comparing Abstractive Summaries Generated by ChatGPT to Real Summaries Through Blinded Reviewers and Text Classification Algorithms](https://mlsanity.com/paper/2303.17650/) (2023)

## Built on by (most cited first, 19 of 19)

- [Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning](https://mlsanity.com/paper/2402.14883/) (2024)
- [StyloAI: Distinguishing AI-Generated Content with Stylometric Analysis](https://mlsanity.com/paper/2405.10129/) (2024)
- [Detecting Machine-Generated Texts: Not Just "AI vs Humans" and Explainability is Complicated](https://mlsanity.com/paper/2406.18259/) (2024)
- [Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem](https://mlsanity.com/paper/2506.08134/) (2025)
- [Linguistic Characteristics of AI-Generated Text: A Survey](https://mlsanity.com/paper/2510.05136/) (2025)
- [Humanizing the Machine: Proxy Attacks to Mislead LLM Detectors](https://mlsanity.com/paper/2410.19230/) (2024)
- [How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses](https://mlsanity.com/paper/2504.13038/) (2025)
- [Limits of LLM Text Detectors in Education](https://mlsanity.com/paper/2508.08096/) (2025)
- [Classification of Human- and AI-Generated Texts for English, French, German, and Spanish](https://mlsanity.com/paper/2312.04882/) (2023)
- [A Certified Robust Watermark For Large Language Models](https://mlsanity.com/paper/2409.19708/) (2024)
- [Tracing Thought: Using Chain-of-Thought Reasoning to Identify the LLM Behind AI-Generated Text](https://mlsanity.com/paper/2504.16913/) (2025)
- [Distinguishing AI-Generated and Human-Written Text Through Psycholinguistic Analysis](https://mlsanity.com/paper/2505.01800/) (2025)
- [An Information-Theoretic Approach for Detecting Edits in AI-Generated Text](https://mlsanity.com/paper/2308.12747/) (2023)
- [Distinguishing Chatbot from Human](https://mlsanity.com/paper/2408.04647/) (2024)
- [SPADE: Structured Prompting Augmentation for Dialogue Enhancement in Machine-Generated Text Detection](https://mlsanity.com/paper/2503.15044/) (2025)
- [Measuring Human Involvement in AI-Generated Text: A Case Study on Academic Writing](https://mlsanity.com/paper/2506.03501/) (2025)
- [Assessing Deanonymization Risks with Stylometry-Assisted LLM Agent](https://mlsanity.com/paper/2602.23079/) (2026)
- [Chameleon: Benchmarking Detection and Backtracking on Commercial-Grade AI-Generated Videos](https://mlsanity.com/paper/2503.06624/) (2025)
- [A Cross-Cultural Assessment of Human Ability to Detect LLM-Generated Fake News about South Africa](https://mlsanity.com/paper/2511.17682/) (2025)

## Abstract (arXiv)

Recently, generative AIs like ChatGPT have become available to the wide public. These tools can for instance be used by students to generate essays or whole theses. But how does a teacher know whether a text is written by a student or an AI? In our work, we explore traditional and new features to (1) detect text generated by AI from scratch and (2) text rephrased by AI. Since we found that classification is more difficult when the AI has been instructed to create the text in a way that a human would not recognize that it was generated by an AI, we also investigate this more advanced case. For our experiments, we produced a new text corpus covering 10 school topics. Our best systems to classify basic and advanced human-generated/AI-generated texts have F1-scores of over 96%. Our best systems for classifying basic and advanced human-generated/AI-rephrased texts have F1-scores of more than 78%. The systems use a combination of perplexity, semantic, list lookup, error-based, readability, AI feedback, and text vector features. Our results show that the new features substantially help to improve the performance of many classifiers. Our best basic text rephrasing detection system even outperforms GPTZero by 183.8% relative in F1-score.