# GPT-4o reads the mind in the eyes

arXiv: https://arxiv.org/abs/2410.22309 · HTML: https://mlsanity.com/paper/2410.22309/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2024.

## Built on by (most cited first, 4 of 4)

- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Do Chains-of-Thoughts of Large Language Models Suffer from Hallucinations, Cognitive Biases, or Phobias in Bayesian Reasoning?](https://mlsanity.com/paper/2503.15268/) (2025)
- [Towards properly implementing Theory of Mind in AI systems: An account of four misconceptions](https://mlsanity.com/paper/2503.16468/) (2025)
- [Theory of Mind in Action: The Instruction Inference Task in Dynamic Human-Agent Collaboration](https://mlsanity.com/paper/2507.02935/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) are capable of reproducing human-like inferences, including inferences about emotions and mental states, from text. Whether this capability extends beyond text to other modalities remains unclear. Humans possess a sophisticated ability to read the mind in the eyes of other people. Here we tested whether this ability is also present in GPT-4o, a multimodal LLM. Using two versions of a widely used theory of mind test, the Reading the Mind in Eyes Test and the Multiracial Reading the Mind in the Eyes Test, we found that GPT-4o outperformed humans in interpreting mental states from upright faces but underperformed humans when faces were inverted. While humans in our sample showed no difference between White and Non-white faces, GPT-4o's accuracy was higher for White than for Non-white faces. GPT-4o's errors were not random but revealed a highly consistent, yet incorrect, processing of mental-state information across trials, with an orientation-dependent error structure that qualitatively differed from that of humans for inverted faces but not for upright faces. These findings highlight how advanced mental state inference abilities and human-like face processing signatures, such as inversion effects, coexist in GPT-4o alongside substantial differences in information processing compared to humans.