# A Turing Test: Are AI Chatbots Behaviorally Similar to Humans?

arXiv: https://arxiv.org/abs/2312.00798 · HTML: https://mlsanity.com/paper/2312.00798/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 41 references that are in the map.
- First posted on arXiv 19 Nov 2023.

## Builds on (most cited first, 7 of 7)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models](https://mlsanity.com/paper/2303.10130/) (2023)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Consciousness in Artificial Intelligence: Insights from the Science of Consciousness](https://mlsanity.com/paper/2308.08708/) (2023)
- [The Foundation Model Transparency Index](https://mlsanity.com/paper/2310.12941/) (2023)
- [The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence](https://mlsanity.com/paper/2201.04200/) (2022)

## Built on by (most cited first, 4 of 4)

- [LLMs with Personalities in Multi-issue Negotiation Games](https://mlsanity.com/paper/2405.05248/) (2024)
- [LLMs Model Non-WEIRD Populations: Experiments with Synthetic Cultural Agents](https://mlsanity.com/paper/2501.06834/) (2025)
- [The Collective Turing Test: Large Language Models Can Generate Realistic Multi-User Discussions](https://mlsanity.com/paper/2511.08592/) (2025)
- [Dynamics of Cognitive Heterogeneity: Investigating Behavioral Biases in Multi-Stage Supply Chains with LLM-Based Simulation](https://mlsanity.com/paper/2604.17220/) (2026)

## Abstract (arXiv)

We administer a Turing Test to AI Chatbots. We examine how Chatbots behave in a suite of classic behavioral games that are designed to elicit characteristics such as trust, fairness, risk-aversion, cooperation, textitetc., as well as how they respond to a traditional Big-5 psychological survey that measures personality traits. ChatGPT-4 exhibits behavioral and personality traits that are statistically indistinguishable from a random human from tens of thousands of human subjects from more than 50 countries. Chatbots also modify their behavior based on previous experience and contexts ``as if'' they were learning from the interactions, and change their behavior in response to different framings of the same strategic situation. Their behaviors are often distinct from average and modal human behaviors, in which case they tend to behave on the more altruistic and cooperative end of the distribution. We estimate that they act as if they are maximizing an average of their own and partner's payoffs.