# Machine Psychology

arXiv: https://arxiv.org/abs/2303.13988 · HTML: https://mlsanity.com/paper/2303.13988/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 169 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 30 of 61)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)

## Built on by (most cited first, 22 of 22)

- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English](https://mlsanity.com/paper/2503.04099/) (2025)
- [Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?](https://mlsanity.com/paper/2505.09614/) (2025)
- [Probing the Preferences of a Language Model: Integrating Verbal and Behavioral Tests of AI Welfare](https://mlsanity.com/paper/2509.07961/) (2025)
- [Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space](https://mlsanity.com/paper/2605.12412/) (2026)
- [Evolutionary ecology of words](https://mlsanity.com/paper/2505.05863/) (2025)
- [AI Through the Human Lens: Investigating Cognitive Theories in Machine Psychology](https://mlsanity.com/paper/2506.18156/) (2025)
- [Applying Psychometrics to Large Language Model Simulated Populations: Recreating the HEXACO Personality Inventory Experiment with Generative Agents](https://mlsanity.com/paper/2508.00742/) (2025)
- [Linguistic traces of stochastic empathy in language models](https://mlsanity.com/paper/2410.01675/) (2024)
- [Evidence of conceptual mastery in the application of rules by Large Language Models](https://mlsanity.com/paper/2503.00992/) (2025)
- [Measuring How LLMs Internalize Human Psychological Concepts: A preliminary analysis](https://mlsanity.com/paper/2506.23055/) (2025)
- [The TEA Nets framework combines AI and cognitive network science to model targets, events and actors in text](https://mlsanity.com/paper/2604.27673/) (2026)
- [Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers](https://mlsanity.com/paper/2606.21037/) (2026)
- [Mixture of Cognitive Experts in Large Vision-Language Models](https://mlsanity.com/paper/2607.10796/) (2026)
- [Measuring Obedience to Authority Across Large Language Models with the Milgram Paradigm](https://mlsanity.com/paper/2608.16177/) (2026)
- [Proxy reliance in large language model decisions is uncalibrated to predictive evidence](https://mlsanity.com/paper/2608.22887/) (2026)
- [Mapping the Emerging Social Science of Large Language Models](https://mlsanity.com/paper/2609.07598/) (2026)
- [The Pushback Paradox: A Two-Probe Diagnostic for Language Model Compliance](https://mlsanity.com/paper/2610.06673/) (2026)
- [Illusory Pattern Perception Drives Spurious Inference in Large Language Models](https://mlsanity.com/paper/2610.07791/) (2026)
- [MASC: A Multi-Agent Self-Calibration Framework with Latent Construct Alignment for Consistent Client Role-Playing in Psychological Counseling](https://mlsanity.com/paper/2610.08250/) (2026)

## Abstract (arXiv)

Large language models (LLMs) show increasingly advanced emergent capabilities and are being incorporated across various societal domains. Understanding their behavior and reasoning abilities therefore holds significant importance. We argue that a fruitful direction for research is engaging LLMs in behavioral experiments inspired by psychology that have traditionally been aimed at understanding human cognition and behavior. In this article, we highlight and summarize theoretical perspectives, experimental paradigms, and computational analysis techniques that this approach brings to the table. It paves the way for a "machine psychology" for generative artificial intelligence (AI) that goes beyond performance benchmarks and focuses instead on computational insights that move us toward a better understanding and discovery of emergent abilities and behavioral patterns in LLMs. We review existing work taking this approach, synthesize best practices, and highlight promising future directions. We also highlight the important caveats of applying methodologies designed for understanding humans to machines. We posit that leveraging tools from experimental psychology to study AI will become increasingly valuable as models evolve to be more powerful, opaque, multi-modal, and integrated into complex real-world settings.