# Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?

arXiv: https://arxiv.org/abs/2308.01284 · HTML: https://mlsanity.com/paper/2308.01284/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 41 references that are in the map.
- First posted on arXiv 2 Aug 2023.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [Undetectable Watermarks for Language Models](https://mlsanity.com/paper/2306.09194/) (2023)
- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation](https://mlsanity.com/paper/2109.13296/) (2021)
- [TweepFake: about Detecting Deepfake Tweets](https://mlsanity.com/paper/2008.00036/) (2020)
- [Human heuristics for AI-generated language are flawed](https://mlsanity.com/paper/2206.07271/) (2022)
- [ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning](https://mlsanity.com/paper/2304.06588/) (2023)
- [Protecting Language Generation Models via Invisible Watermarking](https://mlsanity.com/paper/2302.03162/) (2023)
- [The Turking Test: Can Language Models Understand Instructions?](https://mlsanity.com/paper/2010.11982/) (2020)
- [ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models](https://mlsanity.com/paper/2303.16421/) (2023)

## Built on by (most cited first, 30 of 39)

- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [ConDA: Contrastive Domain Adaptation for AI-generated Text Detection](https://mlsanity.com/paper/2309.03992/) (2023)
- [Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations](https://mlsanity.com/paper/2504.03274/) (2025)
- [User Modeling in the Era of Large Language Models: Current Research and Future Directions](https://mlsanity.com/paper/2312.11518/) (2023)
- [Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs](https://mlsanity.com/paper/2402.10586/) (2024)
- [Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurement](https://mlsanity.com/paper/2410.14259/) (2024)
- [Towards Interpretable Hate Speech Detection using Large Language Model-extracted Rationales](https://mlsanity.com/paper/2403.12403/) (2024)
- [Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection](https://mlsanity.com/paper/2412.11506/) (2024)
- [EAGLE: A Domain Generalization Framework for AI-generated Text Detection](https://mlsanity.com/paper/2403.15690/) (2024)
- [Do LLMs Understand Ambiguity in Text? A Case Study in Open-world Question Answering](https://mlsanity.com/paper/2411.12395/) (2024)
- [Survey on AI-Generated Media Detection: From Non-MLLM to MLLM](https://mlsanity.com/paper/2502.05240/) (2025)
- [Is Contrasting All You Need? Contrastive Learning for the Detection and Attribution of AI-generated Text](https://mlsanity.com/paper/2407.09364/) (2024)
- [OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attribution](https://mlsanity.com/paper/2504.11369/) (2025)
- [Adversarial Text Purification: A Large Language Model Approach for Defense](https://mlsanity.com/paper/2402.06655/) (2024)
- [GPT-4 is judged more human than humans in displaced and inverted Turing tests](https://mlsanity.com/paper/2407.08853/) (2024)
- [Authorship Attribution in Multilingual Machine-Generated Texts](https://mlsanity.com/paper/2508.01656/) (2025)
- [Delving into the Utilisation of ChatGPT in Scientific Publications in Astronomy](https://mlsanity.com/paper/2406.17324/) (2024)
- [ESPERANTO: Evaluating Synthesized Phrases to Enhance Robustness in AI Detection for Text Origination](https://mlsanity.com/paper/2409.14285/) (2024)
- [Decoupling Content and Expression: Two-Dimensional Detection of AI-Generated Text](https://mlsanity.com/paper/2503.00258/) (2025)
- [MAiDE-up: Multilingual Deception Detection of GPT-generated Hotel Reviews](https://mlsanity.com/paper/2404.12938/) (2024)
- [Harnessing Artificial Intelligence to Combat Online Hate: Exploring the Challenges and Opportunities of Large Language Models in Hate Speech Detection](https://mlsanity.com/paper/2403.08035/) (2024)
- [Gen-Review: A Large-scale Dataset of AI-Generated (and Human-written) Peer Reviews](https://mlsanity.com/paper/2510.21192/) (2025)
- [PADBen: A Comprehensive Benchmark for Evaluating AI Text Detectors Against Paraphrase Attacks](https://mlsanity.com/paper/2511.00416/) (2025)
- [TrustAI at SemEval-2024 Task 8: A Comprehensive Analysis of Multi-domain Machine Generated Text Detection Techniques](https://mlsanity.com/paper/2403.16592/) (2024)
- [GigaCheck: Detecting LLM-generated Content via Object-Centric Span Localization](https://mlsanity.com/paper/2410.23728/) (2024)
- [Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection](https://mlsanity.com/paper/2501.03940/) (2025)
- [Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection](https://mlsanity.com/paper/2502.04528/) (2025)
- [M-DAIGT: A Shared Task on Multi-Domain Detection of AI-Generated Text](https://mlsanity.com/paper/2511.11340/) (2025)
- [Beyond Turing: A Comparative Analysis of Approaches for Detecting Machine-Generated Text](https://mlsanity.com/paper/2311.12373/) (2023)
- [Data-Free Model-Related Attacks: Unleashing the Potential of Generative AI](https://mlsanity.com/paper/2501.16671/) (2025)

## Abstract (arXiv)

Large language models (LLMs) such as ChatGPT are increasingly being used for various use cases, including text content generation at scale. Although detection methods for such AI-generated text exist already, we investigate ChatGPT's performance as a detector on such AI-generated text, inspired by works that use ChatGPT as a data labeler or annotator. We evaluate the zero-shot performance of ChatGPT in the task of human-written vs. AI-generated text detection, and perform experiments on publicly available datasets. We empirically investigate if ChatGPT is symmetrically effective in detecting AI-generated or human-written text. Our findings provide insight on how ChatGPT and similar LLMs may be leveraged in automated detection pipelines by simply focusing on solving a specific aspect of the problem and deriving the rest from that solution. All code and data is available at https://github.com/AmritaBh/ChatGPT-as-Detector.