mlsanityData to 7 Oct 2026

People cannot distinguish GPT-4 from a human in a Turing test

arXiv 2405.08007 ↗First posted 9 May 2024cs.HC cs.AIcited by 46

Not on the map. Why?

Abstract (arXiv)

We evaluated 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%). The results provide the first robust empirical demonstration that any artificial system passes an interactive 2-player Turing test. The results have implications for debates around machine intelligence and, more urgently, suggest that deception by current AI systems may go undetected. Analysis of participants' strategies and reasoning suggests that stylistic and socio-emotional factors play a larger role in passing the Turing test than traditional notions of intelligence.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 16 of its 68 references that are in the map. First posted on arXiv 9 May 2024.

Builds on

16 of 68 references.

References · 16

  1. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  2. 2021Ethical and social risks of harm from Language Modelscited by 957
  3. 2023A Watermark for Large Language Modelscited by 667
  4. 2019Defending Against Neural Fake Newscited by 553
  5. 2022Using cognitive psychology to understand GPT-3cited by 389
  6. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  7. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  8. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  9. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  10. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  11. 2023The Vector Grounding Problemcited by 39
  12. 2023Does GPT-4 pass the Turing test?cited by 37
4 more
  1. 2023Artificial Influence: An Analysis Of AI-Driven Persuasioncited by 31
  2. 2023Human or Not? A Gamified Approach to the Turing Testcited by 16
  3. 2023Game of Tones: Faculty detection of GPT-4 generated content in university assessmentscited by 15
  4. 2024Large language models and linguistic intentionalitycited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 22; 2026: 14.

46 papers.

Sort:

By area:

  1. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 50
  2. 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulationscited by 28
  3. 2025Multi-turn Evaluation of Anthropomorphic Behaviours in Large Language Modelscited by 25
  4. 2024From Imitation to Introspection: Probing Self-Consciousness in Language Modelscited by 14
  5. 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 14
  6. 2025TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulationcited by 13
  7. 2025IMPersona: Evaluating Individual Level LM Impersonationcited by 11
  8. 2024Societal Adaptation to Advanced AIcited by 10
  9. 2025When Do Large Language Models Exhibit Unsolicited Deception?cited by 9
  10. 2024GPT-4 is judged more human than humans in displaced and inverted Turing testscited by 7
  11. 2024Transforming Agency. On the mode of existence of Large Language Modelscited by 7
  12. 2024X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agentscited by 6
34 more
  1. 2025Multilingual Performance of a Multimodal Artificial Intelligence System on Multisubject Physics Concept Inventoriescited by 5
  2. 2024Exploring Human-LLM Conversations: Mental Models and the Originator of Toxicitycited by 4
  3. 2025Large Language Models Badly Generalize across Option Length, Problem Types, and Irrelevant Noun Replacementscited by 4
  4. 2025Assessing LLMs in Art Contexts: Critique Generation and Theory of Mind Evaluationcited by 3
  5. 2025Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Riskscited by 3
  6. 2025Love, Lies, and Language Models: Investigating AI's Role in Romance-Baiting Scamscited by 3
  7. 2026Understanding Persuasive Interactions between Generative Social Agents and Humans: The Knowledge-based Persuasion Model (KPM)cited by 3
  8. 2025A Rigorous Turing Test: a Foundation for Evaluating Artificial General Intelligencecited by 2
  9. 2024Probing for Consciousness in Machinescited by 1
  10. 2025ChatGPT-4 and other LLMs in the Turing Test: A Critical Analysiscited by 1
  11. 2025ChatGPT-generated texts show authorship traits that identify them as non-humancited by 1
  12. 2026Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interactioncited by 1
  13. 2024How to Measure the Intelligence of Large Language Models?cited by 0
  14. 2024AI-Driven Agents with Prompts Designed for High Agreeableness Increase the Likelihood of Being Mistaken for a Human in the Turing Testcited by 0
  15. 2025Evidence of conceptual mastery in the application of rules by Large Language Modelscited by 0
  16. 2025React to This (RTT): A Nonverbal Turing Test for Embodied AIcited by 0
  17. 2025Assessing Consciousness-Related Behaviors in Large Language Models Using the Maze Testcited by 0
  18. 2025Normality and the Turing Testcited by 0
  19. 2025Language models as tools for investigating the distinction between possible and impossible natural languagescited by 0
  20. 2025Large language models have learned to use languagecited by 0
  21. 2025Large language models are not about natural languagecited by 0
  22. 2025Linguists should learn to love speech-based deep learning modelscited by 0
  23. 2026Are Language Models Models?cited by 0
  24. 2026RCTs for Frontier AI Governance: Methodological Challenges and Solutions for Human Uplift Studiescited by 0
  25. 2026Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated Probabilitiescited by 0
  26. 2026You Can't Fight in Here! This is BBS!cited by 0
  27. 2026Heterogeneity in Formal Linguistic Competence of Language Models: Is Data the Real Bottleneck?cited by 0
  28. 2026Not-So-Strange Love: Language Models and Generative Linguistic Theories are More Compatible than They Appearcited by 0
  29. 2026Generalizing the Turing Test to Interactive Agentscited by 0
  30. 2026Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthropomorphism, and Maximscited by 0
  31. 2026Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Promptscited by 0
  32. 2026Identity from the Outside: A Conceptual Framework and Research Program for AI Personality Clonescited by 0
  33. 2026Detecting AI Impostors: How Do Middle Schoolers Identify LLM Agents in a Live Collaborative Setting?cited by 0
  34. 2026Synthetic Hospital: An Open, Verifiable, Physician-Validated Longitudinal EHR Benchmarkcited by 0