# Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina

arXiv: https://arxiv.org/abs/2410.19599 · HTML: https://mlsanity.com/paper/2410.19599/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 36 of its 73 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 36)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"](https://mlsanity.com/paper/2309.12288/) (2023)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [Don't Make Your LLM an Evaluation Benchmark Cheater](https://mlsanity.com/paper/2311.01964/) (2023)
- [Let's Think Dot by Dot: Hidden Computation in Transformer Language Models](https://mlsanity.com/paper/2404.15758/) (2024)
- [The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation](https://mlsanity.com/paper/2301.01768/) (2023)
- [Can Large Language Models Infer Causation from Correlation?](https://mlsanity.com/paper/2306.05836/) (2023)
- [Large Language Models Understand and Can be Enhanced by Emotional Stimuli](https://mlsanity.com/paper/2307.11760/) (2023)
- [Language Models Meet World Models: Embodied Experiences Enhance Language Models](https://mlsanity.com/paper/2305.10626/) (2023)
- [Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples](https://mlsanity.com/paper/2305.15269/) (2023)
- [Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs](https://mlsanity.com/paper/2402.17649/) (2024)
- [Large Language Models Are Not Strong Abstract Reasoners](https://mlsanity.com/paper/2305.19555/) (2023)
- [Changing Answer Order Can Decrease MMLU Accuracy](https://mlsanity.com/paper/2406.19470/) (2024)
- [LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond](https://mlsanity.com/paper/2305.14540/) (2023)

## Built on by (most cited first, 29 of 29)

- [LLM Social Simulations Are a Promising Research Method](https://mlsanity.com/paper/2504.02234/) (2025)
- [Finetuning LLMs for Human Behavior Prediction in Social Science Experiments](https://mlsanity.com/paper/2509.05830/) (2025)
- [Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions](https://mlsanity.com/paper/2502.20897/) (2025)
- [The simulation of judgment in LLMs](https://mlsanity.com/paper/2502.04426/) (2025)
- [It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics](https://mlsanity.com/paper/2506.02873/) (2025)
- [A validity-guided workflow for robust large language model research in psychology](https://mlsanity.com/paper/2507.04491/) (2025)
- [Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiences](https://mlsanity.com/paper/2510.06105/) (2025)
- [PredictaBoard: Benchmarking LLM Score Predictability](https://mlsanity.com/paper/2502.14445/) (2025)
- [A Multi-Task Evaluation of LLMs' Processing of Academic Text Input](https://mlsanity.com/paper/2508.11779/) (2025)
- [PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactions](https://mlsanity.com/paper/2509.07370/) (2025)
- [The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study](https://mlsanity.com/paper/2605.20767/) (2026)
- [Static network structure cannot stabilize cooperation among Large Language Model agents](https://mlsanity.com/paper/2411.10294/) (2024)
- [Reasoning and the Trusting Behavior of DeepSeek and GPT: An Experiment Revealing Hidden Fault Lines in Large Language Models](https://mlsanity.com/paper/2502.12825/) (2025)
- [Improving the Distributional Alignment of LLMs using Supervision](https://mlsanity.com/paper/2507.00439/) (2025)
- [Robustness is Important: Limitations of LLMs for Data Fitting](https://mlsanity.com/paper/2508.19563/) (2025)
- [LISTEN to Your Preferences: An LLM Framework for Multi-Objective Selection](https://mlsanity.com/paper/2510.25799/) (2025)
- [Reliable agent engineering should integrate machine-compatible organizational principles](https://mlsanity.com/paper/2512.07665/) (2025)
- [The Collapse of Heterogeneity in Silicon Philosophers](https://mlsanity.com/paper/2604.23575/) (2026)
- [The LLM Wears Prada: Analysing Gender Bias and Stereotypes through Online Shopping Data](https://mlsanity.com/paper/2504.01951/) (2025)
- [Statistical Foundations of LLM-based A/B Testing: A Surrogacy Framework for Human Causal Inference](https://mlsanity.com/paper/2606.17165/) (2026)
- [Aplaud: Adaptive Personalized Low-Rank Decomposition for User-Specific LLM](https://mlsanity.com/paper/2609.04738/) (2026)
- [Natural Language Generation](https://mlsanity.com/paper/2503.16728/) (2025)
- [Synthetic Data Generation for Screen Time and App Usage](https://mlsanity.com/paper/2509.13892/) (2025)
- [Estimating the Error of Large Language Models at Pairwise Text Comparison](https://mlsanity.com/paper/2510.22219/) (2025)
- [Knowing Your Uncertainty -- On the application of LLM in social sciences](https://mlsanity.com/paper/2512.05461/) (2025)
- [Improving Behavioral Alignment in LLM Social Simulations via Context Formation and Navigation](https://mlsanity.com/paper/2601.01546/) (2026)
- [To Words and Beyond: Probing Large Language Models for Sentence-Level Psycholinguistic Norms of Memorability and Reading Times](https://mlsanity.com/paper/2603.12105/) (2026)
- [Will Scaling Improve Social Simulation with LLMs?](https://mlsanity.com/paper/2607.02464/) (2026)
- [Proxy-Validated LLM UX Micro-Simulations: An Artifact-First Protocol for Early-Stage Decision Support](https://mlsanity.com/paper/2608.13563/) (2026)

## Abstract (arXiv)

Recent studies suggest large language models (LLMs) can exhibit human-like reasoning, aligning with human behavior in economic experiments, surveys, and political discourse. This has led many to propose that LLMs can be used as surrogates or simulations for humans in social science research. However, LLMs differ fundamentally from humans, relying on probabilistic patterns, absent the embodied experiences or survival objectives that shape human cognition. We assess the reasoning depth of LLMs using the 11-20 money request game. Nearly all advanced approaches fail to replicate human behavior distributions across many models. Causes of failure are diverse and unpredictable, relating to input language, roles, and safeguarding. These results advise caution when using LLMs to study human behavior or as surrogates or simulations.