# Mind meets machine: Unravelling GPT-4's cognitive psychology

arXiv: https://arxiv.org/abs/2303.11436 · HTML: https://mlsanity.com/paper/2303.11436/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [WinoGrande: An Adversarial Winograd Schema Challenge at Scale](https://mlsanity.com/paper/1907.10641/) (2019)
- [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022)
- [Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks](https://mlsanity.com/paper/2210.00400/) (2022)
- [AI for Next Generation Computing: Emerging Trends and Future Directions](https://mlsanity.com/paper/2203.04159/) (2022)

## Built on by (most cited first, 30 of 31)

- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities](https://mlsanity.com/paper/2305.13168/) (2023)
- [PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety](https://mlsanity.com/paper/2401.11880/) (2024)
- [Strategic Behavior of Large Language Models: Game Structure vs. Contextual Framing](https://mlsanity.com/paper/2309.05898/) (2023)
- [CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering](https://mlsanity.com/paper/2403.16008/) (2024)
- [ChatCounselor: A Large Language Models for Mental Health Support](https://mlsanity.com/paper/2309.15461/) (2023)
- [DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy](https://mlsanity.com/paper/2310.18659/) (2023)
- [Interpreting Pretrained Language Models via Concept Bottlenecks](https://mlsanity.com/paper/2311.05014/) (2023)
- [AI for social science and social science of AI: A Survey](https://mlsanity.com/paper/2401.11839/) (2024)
- [LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Output](https://mlsanity.com/paper/2411.06032/) (2024)
- [Transformative Effects of ChatGPT on Modern Education: Emerging Era of AI Chatbots](https://mlsanity.com/paper/2306.03823/) (2023)
- [Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review](https://mlsanity.com/paper/2401.01519/) (2024)
- [An Appraisal-Based Chain-Of-Emotion Architecture for Affective Language Model Game Agents](https://mlsanity.com/paper/2309.05076/) (2023)
- [What Should Data Science Education Do with Large Language Models?](https://mlsanity.com/paper/2307.02792/) (2023)
- [ChatGPT: Vision and Challenges](https://mlsanity.com/paper/2305.15323/) (2023)
- [SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabia](https://mlsanity.com/paper/2503.17485/) (2025)
- [CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models](https://mlsanity.com/paper/2401.08438/) (2024)
- [Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach](https://mlsanity.com/paper/2310.08172/) (2023)
- [Unveiling Narrative Reasoning Limits of Large Language Models with Trope in Movie Synopses](https://mlsanity.com/paper/2409.14324/) (2024)
- [Towards a Psychology of Machines: Large Language Models Predict Human Memory](https://mlsanity.com/paper/2403.05152/) (2024)
- [Professional Certification Benchmark Dataset: The First 500 Jobs For Large Language Models](https://mlsanity.com/paper/2305.05377/) (2023)
- [Empathy-R1: A Chain-of-Empathy and Reinforcement Learning Framework for Long-Form Mental Health Support](https://mlsanity.com/paper/2509.14851/) (2025)
- [Evaluating the capability of large language models to personalize science texts for diverse middle-school-age learners](https://mlsanity.com/paper/2408.05204/) (2024)
- [Judgment of Learning: A Human Ability Beyond Generative Artificial Intelligence](https://mlsanity.com/paper/2410.13392/) (2024)
- [Evaluating AI Vocational Skills Through Professional Testing](https://mlsanity.com/paper/2312.10603/) (2023)
- [Representation in large language models](https://mlsanity.com/paper/2501.00885/) (2025)

## Abstract (arXiv)

Cognitive psychology delves on understanding perception, attention, memory, language, problem-solving, decision-making, and reasoning. Large language models (LLMs) are emerging as potent tools increasingly capable of performing human-level tasks. The recent development in the form of GPT-4 and its demonstrated success in tasks complex to humans exam and complex problems has led to an increased confidence in the LLMs to become perfect instruments of intelligence. Although GPT-4 report has shown performance on some cognitive psychology tasks, a comprehensive assessment of GPT-4, via the existing well-established datasets is required. In this study, we focus on the evaluation of GPT-4's performance on a set of cognitive psychology datasets such as CommonsenseQA, SuperGLUE, MATH and HANS. In doing so, we understand how GPT-4 processes and integrates cognitive psychology with contextual information, providing insight into the underlying cognitive processes that enable its ability to generate the responses. We show that GPT-4 exhibits a high level of accuracy in cognitive psychology tasks relative to the prior state-of-the-art models. Our results strengthen the already available assessments and confidence on GPT-4's cognitive psychology abilities. It has significant potential to revolutionize the field of AI, by enabling machines to bridge the gap between human and machine reasoning.