# ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text

arXiv: https://arxiv.org/abs/2301.13852 · HTML: https://mlsanity.com/paper/2301.13852/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 13 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Learning Deep Transformer Models for Machine Translation](https://mlsanity.com/paper/1906.01787/) (2019)
- [Phrase-Based & Neural Unsupervised Machine Translation](https://mlsanity.com/paper/1804.07755/) (2018)
- [Tensor2Tensor for Neural Machine Translation](https://mlsanity.com/paper/1803.07416/) (2018)

## Built on by (most cited first, 30 of 85)

- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Who Wrote this Code? Watermarking for Code Generation](https://mlsanity.com/paper/2305.15060/) (2023)
- [Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness](https://mlsanity.com/paper/2304.11633/) (2023)
- [DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text](https://mlsanity.com/paper/2305.17359/) (2023)
- [A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://mlsanity.com/paper/2305.18486/) (2023)
- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [Multiscale Positive-Unlabeled Detection of AI-Generated Texts](https://mlsanity.com/paper/2305.18149/) (2023)
- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Towards Codable Watermarking for Injecting Multi-bits Information to LLMs](https://mlsanity.com/paper/2307.15992/) (2023)
- [Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedy](https://mlsanity.com/paper/2307.13808/) (2023)
- [ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models](https://mlsanity.com/paper/2304.07666/) (2023)
- [Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions](https://mlsanity.com/paper/2308.02312/) (2023)
- [GeoGPT: Understanding and Processing Geospatial Tasks through An Autonomous GPT](https://mlsanity.com/paper/2307.07930/) (2023)
- [CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts](https://mlsanity.com/paper/2304.12008/) (2023)
- [HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis](https://mlsanity.com/paper/2305.18226/) (2023)
- [Generative AI Text Classification using Ensemble LLM Approaches](https://mlsanity.com/paper/2309.07755/) (2023)
- [Differentiate ChatGPT-generated and Human-written Medical Texts](https://mlsanity.com/paper/2304.11567/) (2023)
- [Bot or Human? Detecting ChatGPT Imposters with A Single Question](https://mlsanity.com/paper/2305.06424/) (2023)
- [From Text to Source: Results in Detecting Large Language Model-Generated Content](https://mlsanity.com/paper/2309.13322/) (2023)
- [How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts](https://mlsanity.com/paper/2310.05095/) (2023)
- [AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees](https://mlsanity.com/paper/2510.01268/) (2025)
- [To ChatGPT, or not to ChatGPT: That is the question!](https://mlsanity.com/paper/2304.01487/) (2023)
- [Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT](https://mlsanity.com/paper/2308.05341/) (2023)
- [DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection](https://mlsanity.com/paper/2305.12519/) (2023)
- [Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directions](https://mlsanity.com/paper/2307.14107/) (2023)
- [Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis](https://mlsanity.com/paper/2304.05534/) (2023)
- [The Age of Synthetic Realities: Challenges and Opportunities](https://mlsanity.com/paper/2306.11503/) (2023)

## Abstract (arXiv)

ChatGPT has the ability to generate grammatically flawless and seemingly-human replies to different types of questions from various domains. The number of its users and of its applications is growing at an unprecedented rate. Unfortunately, use and abuse come hand in hand. In this paper, we study whether a machine learning model can be effectively trained to accurately distinguish between original human and seemingly human (that is, ChatGPT-generated) text, especially when this text is short. Furthermore, we employ an explainable artificial intelligence framework to gain insight into the reasoning behind the model trained to differentiate between ChatGPT-generated and human-generated text. The goal is to analyze model's decisions and determine if any specific patterns or characteristics can be identified. Our study focuses on short online reviews, conducting two experiments comparing human-generated and ChatGPT-generated text. The first experiment involves ChatGPT text generated from custom queries, while the second experiment involves text generated by rephrasing original human-generated reviews. We fine-tune a Transformer-based model and use it to make predictions, which are then explained using SHAP. We compare our model with a perplexity score-based approach and find that disambiguation between human and ChatGPT-generated reviews is more challenging for the ML model when using rephrased text. However, our proposed approach still achieves an accuracy of 79%. Using explainability, we observe that ChatGPT's writing is polite, without specific details, using fancy and atypical vocabulary, impersonal, and typically it does not express feelings.