mlsanityData to 7 Oct 2026

A Statistical Turing Test for Generative Models

arXiv 2309.08913 ↗First posted 16 Sep 2023cs.AI cs.CL cs.CYcited by 8

Not on the map. Why?

Abstract (arXiv)

The emergence of human-like abilities of AI systems for content generation in domains such as text, audio, and vision has prompted the development of classifiers to determine whether content originated from a human or a machine. Implicit in these efforts is an assumption that the generation properties of a human are different from that of the machine. In this work, we provide a framework in the language of statistical pattern recognition that quantifies the difference between the distributions of human and machine-generated content conditioned on an evaluation context. We describe current methods in the context of the framework and demonstrate how to use the framework to evaluate the progression of generative models towards human-like capabilities, among many axes of analysis.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 22 of its 46 references that are in the map. First posted on arXiv 16 Sep 2023.

Builds on

22 of 46 references.

References · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  9. 2018Hierarchical Neural Story Generationcited by 1,182
  10. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  11. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  12. 2023A Watermark for Large Language Modelscited by 667
10 more
  1. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  2. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
  3. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  4. 2020Media Forensics and DeepFakes: an overviewcited by 208
  5. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  6. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
  7. 2019Deep Learning for Deepfakes Creation and Detection: A Surveycited by 102
  8. 2019Real or Fake? Learning to Discriminate Machine from Human Generated Textcited by 98
  9. 2023Comparing Foundation Models using Data Kernelscited by 14
  10. 2020A partition-based similarity for classification distributionscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 2; 2026: 1.

8 papers.

  1. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 255
  2. 2024Tracking the perspectives of interacting language modelscited by 14
  3. 2024Statistical inference on black-box generative models in the data kernel perspective spacecited by 9
  4. 2024X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agentscited by 6
  5. 2024Investigating social alignment via mirroring in a system of interacting language modelscited by 5
  6. 2025Toward a digital twin of U.S. Congresscited by 1
  7. 2025Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbationscited by 1
  8. 2026When prompt perturbations break your A/B test: A valid statistical test for generative surveyingcited by 1