# Does GPT-4 pass the Turing test?

arXiv: https://arxiv.org/abs/2310.20216 · HTML: https://mlsanity.com/paper/2310.20216/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 33 references that are in the map.
- First posted on arXiv 31 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)

## Built on by (most cited first, 30 of 37)

- [Large Language Models Assume People are More Rational than We Really are](https://mlsanity.com/paper/2406.17055/) (2024)
- [People cannot distinguish GPT-4 from a human in a Turing test](https://mlsanity.com/paper/2405.08007/) (2024)
- [Large Language Models Pass the Turing Test](https://mlsanity.com/paper/2503.23674/) (2025)
- [TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification](https://mlsanity.com/paper/2402.12991/) (2024)
- [Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training](https://mlsanity.com/paper/2506.01732/) (2025)
- [How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHO](https://mlsanity.com/paper/2404.13957/) (2024)
- [Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models](https://mlsanity.com/paper/2412.17128/) (2024)
- [TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation](https://mlsanity.com/paper/2510.25536/) (2025)
- [Thinking beyond the anthropomorphic paradigm benefits LLM research](https://mlsanity.com/paper/2502.09192/) (2025)
- [Helmsman of the Masses? Evaluate the Opinion Leadership of Large Language Models in the Werewolf Game](https://mlsanity.com/paper/2404.01602/) (2024)
- [The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment](https://mlsanity.com/paper/2412.16468/) (2024)
- [The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making](https://mlsanity.com/paper/2410.07304/) (2024)
- [IMPersona: Evaluating Individual Level LM Impersonation](https://mlsanity.com/paper/2504.04332/) (2025)
- [GPT-4 is judged more human than humans in displaced and inverted Turing tests](https://mlsanity.com/paper/2407.08853/) (2024)
- [SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent](https://mlsanity.com/paper/2410.14152/) (2024)
- [Different types of syntactic agreement recruit the same units within large language models](https://mlsanity.com/paper/2512.03676/) (2025)
- [Revealing emergent human-like conceptual representations from language prediction](https://mlsanity.com/paper/2501.12547/) (2025)
- [Digital Player: Evaluating Large Language Models based Human-like Agent in Games](https://mlsanity.com/paper/2502.20807/) (2025)
- [Philosophy of Cognitive Science in the Age of Deep Learning](https://mlsanity.com/paper/2405.04048/) (2024)
- [Are You Human? An Adversarial Benchmark to Expose LLMs](https://mlsanity.com/paper/2410.09569/) (2024)
- [Fostering Natural Conversation in Large Language Models with NICO: a Natural Interactive COnversation dataset](https://mlsanity.com/paper/2408.09330/) (2024)
- [The State Of TTS: A Case Study with Human Fooling Rates](https://mlsanity.com/paper/2508.04179/) (2025)
- [Dynamic Normativity: Necessary and Sufficient Conditions for Value Alignment](https://mlsanity.com/paper/2406.11039/) (2024)
- [Assessing LLMs in Art Contexts: Critique Generation and Theory of Mind Evaluation](https://mlsanity.com/paper/2504.12805/) (2025)
- [Love, Lies, and Language Models: Investigating AI's Role in Romance-Baiting Scams](https://mlsanity.com/paper/2512.16280/) (2025)
- [A Rigorous Turing Test: a Foundation for Evaluating Artificial General Intelligence](https://mlsanity.com/paper/2501.17629/) (2025)
- [How Open Must Language Models be to Enable Reliable Scientific Inference?](https://mlsanity.com/paper/2603.26539/) (2026)
- [The Voice: Lessons on Trustworthy Conversational Agents from "Dune"](https://mlsanity.com/paper/2407.18928/) (2024)
- [Advancements and limitations of LLMs in replicating human color-word associations](https://mlsanity.com/paper/2411.02116/) (2024)
- [ChatGPT-4 and other LLMs in the Turing Test: A Critical Analysis](https://mlsanity.com/paper/2503.06551/) (2025)

## Abstract (arXiv)

We evaluated GPT-4 in a public online Turing test. The best-performing GPT-4 prompt passed in 49.7% of games, outperforming ELIZA (22%) and GPT-3.5 (20%), but falling short of the baseline set by human participants (66%). Participants' decisions were based mainly on linguistic style (35%) and socioemotional traits (27%), supporting the idea that intelligence, narrowly conceived, is not sufficient to pass the Turing test. Participant knowledge about LLMs and number of games played positively correlated with accuracy in detecting AI, suggesting learning and practice as possible strategies to mitigate deception. Despite known limitations as a test of intelligence, we argue that the Turing test continues to be relevant as an assessment of naturalistic communication and deception. AI models with the ability to masquerade as humans could have widespread societal consequences, and we analyse the effectiveness of different strategies and criteria for judging humanlikeness.