# People cannot distinguish GPT-4 from a human in a Turing test

arXiv: https://arxiv.org/abs/2405.08007 · HTML: https://mlsanity.com/paper/2405.08007/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 68 references that are in the map.
- First posted on arXiv 9 May 2024.

## Builds on (most cited first, 16 of 16)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [AI Deception: A Survey of Examples, Risks, and Potential Solutions](https://mlsanity.com/paper/2308.14752/) (2023)
- [The Debate Over Understanding in AI's Large Language Models](https://mlsanity.com/paper/2210.13966/) (2022)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [The Vector Grounding Problem](https://mlsanity.com/paper/2304.01481/) (2023)
- [Does GPT-4 pass the Turing test?](https://mlsanity.com/paper/2310.20216/) (2023)
- [Artificial Influence: An Analysis Of AI-Driven Persuasion](https://mlsanity.com/paper/2303.08721/) (2023)
- [Game of Tones: Faculty detection of GPT-4 generated content in university assessments](https://mlsanity.com/paper/2305.18081/) (2023)
- [Human or Not? A Gamified Approach to the Turing Test](https://mlsanity.com/paper/2305.20010/) (2023)
- [Large language models and linguistic intentionality](https://mlsanity.com/paper/2404.09576/) (2024)

## Built on by (most cited first, 30 of 48)

- [How Linguistics Learned to Stop Worrying and Love the Language Models](https://mlsanity.com/paper/2501.17047/) (2025)
- [Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations](https://mlsanity.com/paper/2504.03274/) (2025)
- [Multi-turn Evaluation of Anthropomorphic Behaviours in Large Language Models](https://mlsanity.com/paper/2502.07077/) (2025)
- [Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models](https://mlsanity.com/paper/2412.17128/) (2024)
- [TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation](https://mlsanity.com/paper/2510.25536/) (2025)
- [From Imitation to Introspection: Probing Self-Consciousness in Language Models](https://mlsanity.com/paper/2410.18819/) (2024)
- [IMPersona: Evaluating Individual Level LM Impersonation](https://mlsanity.com/paper/2504.04332/) (2025)
- [Societal Adaptation to Advanced AI](https://mlsanity.com/paper/2405.10295/) (2024)
- [When Do Large Language Models Exhibit Unsolicited Deception?](https://mlsanity.com/paper/2504.00285/) (2025)
- [GPT-4 is judged more human than humans in displaced and inverted Turing tests](https://mlsanity.com/paper/2407.08853/) (2024)
- [Transforming Agency. On the mode of existence of Large Language Models](https://mlsanity.com/paper/2407.10735/) (2024)
- [X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agents](https://mlsanity.com/paper/2408.09853/) (2024)
- [Multilingual Performance of a Multimodal Artificial Intelligence System on Multisubject Physics Concept Inventories](https://mlsanity.com/paper/2501.06143/) (2025)
- [Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Risks](https://mlsanity.com/paper/2505.19806/) (2025)
- [Exploring Human-LLM Conversations: Mental Models and the Originator of Toxicity](https://mlsanity.com/paper/2407.05977/) (2024)
- [Large Language Models Badly Generalize across Option Length, Problem Types, and Irrelevant Noun Replacements](https://mlsanity.com/paper/2502.12459/) (2025)
- [Assessing LLMs in Art Contexts: Critique Generation and Theory of Mind Evaluation](https://mlsanity.com/paper/2504.12805/) (2025)
- [Love, Lies, and Language Models: Investigating AI's Role in Romance-Baiting Scams](https://mlsanity.com/paper/2512.16280/) (2025)
- [Understanding Persuasive Interactions between Generative Social Agents and Humans: The Knowledge-based Persuasion Model (KPM)](https://mlsanity.com/paper/2602.11483/) (2026)
- [A Rigorous Turing Test: a Foundation for Evaluating Artificial General Intelligence](https://mlsanity.com/paper/2501.17629/) (2025)
- [Probing for Consciousness in Machines](https://mlsanity.com/paper/2411.16262/) (2024)
- [ChatGPT-4 and other LLMs in the Turing Test: A Critical Analysis](https://mlsanity.com/paper/2503.06551/) (2025)
- [ChatGPT-generated texts show authorship traits that identify them as non-human](https://mlsanity.com/paper/2508.16385/) (2025)
- [Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction](https://mlsanity.com/paper/2602.24080/) (2026)
- [How to Measure the Intelligence of Large Language Models?](https://mlsanity.com/paper/2407.20828/) (2024)
- [AI-Driven Agents with Prompts Designed for High Agreeableness Increase the Likelihood of Being Mistaken for a Human in the Turing Test](https://mlsanity.com/paper/2411.13749/) (2024)
- [Evidence of conceptual mastery in the application of rules by Large Language Models](https://mlsanity.com/paper/2503.00992/) (2025)
- [React to This (RTT): A Nonverbal Turing Test for Embodied AI](https://mlsanity.com/paper/2507.10812/) (2025)
- [Assessing Consciousness-Related Behaviors in Large Language Models Using the Maze Test](https://mlsanity.com/paper/2508.16705/) (2025)
- [Normality and the Turing Test](https://mlsanity.com/paper/2508.21382/) (2025)

## Abstract (arXiv)

We evaluated 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%). The results provide the first robust empirical demonstration that any artificial system passes an interactive 2-player Turing test. The results have implications for debates around machine intelligence and, more urgently, suggest that deception by current AI systems may go undetected. Analysis of participants' strategies and reasoning suggests that stylistic and socio-emotional factors play a larger role in passing the Turing test than traditional notions of intelligence.