mlsanityData to 14 Oct 2026

GPT detectors are biased against non-native English writers

arXiv 2304.02819 ↗First posted 6 Apr 2023cs.CL cs.AI cs.HCcited by 178

Not on the map. Why?

Cited by 22 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The rapid adoption of generative language models has brought about substantial advancements in digital communication, while simultaneously raising concerns regarding the potential misuse of AI-generated content. Although numerous detection methods have been proposed to differentiate between AI and human-generated content, the fairness and robustness of these detectors remain underexplored. In this study, we evaluate the performance of several widely-used GPT detectors using writing samples from native and non-native English writers. Our findings reveal that these detectors consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are accurately identified. Furthermore, we demonstrate that simple prompting strategies can not only mitigate this bias but also effectively bypass GPT detectors, suggesting that GPT detectors may unintentionally penalize writers with constrained linguistic expressions. Our results call for a broader conversation about the ethical implications of deploying ChatGPT content detectors and caution against their use in evaluative or educational settings, particularly when they may inadvertently penalize or exclude non-native English speakers from the global discourse. The published version of this study can be accessed at: www.cell.com/patterns/fulltext/S2666-3899(23)00130-7

Key facts. Cited by 178 papers in the map (data through 14 Oct 2026). Builds on 13 of its 46 references that are in the map. First posted on arXiv 6 Apr 2023.

Builds on

13 of 46 references.

References · 13

  1. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  2. 2023A Watermark for Large Language Modelscited by 725
  3. 2019Release Strategies and the Social Impacts of Language Modelscited by 513
  4. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 496
  5. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 429
  6. 2023Can AI-Generated Text be Reliably Detected?cited by 386
  7. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  8. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 261
  9. 2020Automatic Detection of Machine Generated Text: A Critical Surveycited by 166
  10. 2020TweepFake: about Detecting Deepfake Tweetscited by 117
  11. 2022Evaluating Human-Language Model Interactioncited by 92
  12. 2022Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methodscited by 79
1 more
  1. 2022Watermarking Pre-trained Language Models with Backdooringcited by 50

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 42; 2024: 53; 2025: 38; 2026: 45.

178 papers.

Sort:

By area:

  1. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 367
  2. 2023Provable Robust Watermarking for AI-Generated Textcited by 299
  3. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 287
  4. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 263
  5. 2023Undetectable Watermarks for Language Modelscited by 226
  6. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 190
  7. 2023On the Reliability of Watermarks for Large Language Modelscited by 189
  8. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 164
  9. 2024RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectorscited by 163
  10. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  11. 2023MAGE: Machine-generated Text Detection in the Wildcited by 132
  12. 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 129
38 more
  1. 2023M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detectioncited by 121
  2. 2024Mapping the Increasing Use of LLMs in Scientific Paperscited by 118
  3. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 112
  4. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 105
  5. 2023Ghostbuster: Detecting Text Ghostwritten by Large Language Modelscited by 91
  6. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 91
  7. 2023On the Possibilities of AI-Generated Text Detectioncited by 88
  8. 2023Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Modelscited by 87
  9. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  10. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 82
  11. 2023Watermarking Text Generated by Black-Box Language Modelscited by 76
  12. 2023A Survey on Detection of LLMs-Generated Contentcited by 71
  13. 2024A Statistical Framework of Watermarks for Large Language Models: Pivot, Detection Efficiency and Optimal Rulescited by 57
  14. 2024Raidar: geneRative AI Detection viA Rewritingcited by 52
  15. 2024SoK: Watermarking for AI-Generated Contentcited by 51
  16. 2025The Widespread Adoption of Large Language Model-Assisted Writing Across Societycited by 51
  17. 2023CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTractscited by 40
  18. 2024Securing Large Language Models: Threats, Vulnerabilities and Responsible Practicescited by 37
  19. 2023Anatomy of an AI-powered malicious social botnetcited by 36
  20. 2024Training-free LLM-generated Text Detection by Mining Token Probability Sequencescited by 35
  21. 2024Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Modelscited by 34
  22. 2023Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Modelscited by 28
  23. 2023Towards Possibilities & Impossibilities of AI-generated Text Detection: A Surveycited by 28
  24. 2023Token Prediction as Implicit Classification to Identify LLM-Generated Textcited by 27
  25. 2023WatME: Towards Lossless Watermarking Through Lexical Redundancycited by 27
  26. 2024Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attackcited by 27
  27. 2024Detecting AI-Generated Text: Factors Influencing Detectability with Current Methodscited by 23
  28. 2024Permute-and-Flip: An optimally stable and watermarkable decoder for LLMscited by 22
  29. 2024The Rise of AI-Generated Content in Wikipediacited by 22
  30. 2025AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guaranteescited by 21
  31. 2024Is Your Paper Being Reviewed by an LLM? Investigating AI Text Detectability in Peer Reviewcited by 20
  32. 2025DAMAGE: Detecting Adversarially Modified AI Generated Textcited by 20
  33. 2024Beyond Binary: Towards Fine-Grained LLM-Generated Text Detection via Role Recognition and Involvement Measurementcited by 17
  34. 2023Game of Tones: Faculty detection of GPT-4 generated content in university assessmentscited by 16
  35. 2023Friend or Foe? Exploring the Implications of Large Language Models on the Science Systemcited by 16
  36. 2024Towards Better Statistical Understanding of Watermarking LLMscited by 16
  37. 2023Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Textcited by 15
  38. 2023Challenges and Remedies to Privacy and Security in AIGC: Exploring the Potential of Privacy Computing, Blockchain, and Beyondcited by 14

50 most cited of 178 All 178 (JSON)