Thinking beyond the anthropomorphic paradigm benefits LLM research
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Anthropomorphism, or the attribution of human traits to technology, is an automatic and unconscious response that occurs even in those with advanced technical expertise. In this position paper, we analyze hundreds of thousands of research articles to present empirical evidence of the prevalence and growth of anthropomorphic terminology in research on large language models (LLMs). We argue for challenging the deeper assumptions reflected in this terminology -- which, though often useful, may inadvertently constrain LLM development -- and broadening beyond them to open new pathways for understanding and improving LLMs. Specifically, we identify and examine five anthropomorphic assumptions that shape research across the LLM development lifecycle. For each assumption (e.g., that LLMs must use natural language for reasoning, or that they should be evaluated on benchmarks originally meant for humans), we demonstrate empirical, non-anthropomorphic alternatives that remain under-explored yet offer promising directions for LLM research and development.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 53 of its 105 references that are in the map. First posted on arXiv 13 Feb 2025.
Builds on
53 of 105 references.
Closest ancestors · 1
Same direction, newest first
Other references · 52
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
- 2023A Survey on Evaluation of Large Language Modelscited by 1,227
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
38 more
- 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
- 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
- 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
- 2024Alignment faking in large language modelscited by 316
- 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
- 2023Role-Play with Large Language Modelscited by 290
- 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
- 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
- 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
- 2022Talking About Large Language Modelscited by 174
- 2022Language Models as Agent Modelscited by 131
- 2023Eliciting Human Preferences with Language Modelscited by 90
- 2022Mapping global dynamics of benchmark creation and saturation in artificial intelligencecited by 71
- 2023Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfacescited by 71
- 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
- 2022In conversation with Artificial Intelligence: aligning language models with human valuescited by 61
- 2020Conceptual Metaphors Impact Perceptions of Human-AI Collaborationcited by 47
- 2024Beyond Preferences in AI Alignmentcited by 46
- 2022The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligencecited by 42
- 2024Confabulation: The Surprising Value of Large Language Model Hallucinationscited by 42
- 2024Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluationcited by 41
- 2024Cocoa: Co-Planning and Co-Execution with AI Agentscited by 38
- 2023Does GPT-4 pass the Turing test?cited by 37
- 2022What do tokens know about their characters and how do they know it?cited by 36
- 2024Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Modelscited by 36
- 2024AnthroScore: A Computational Linguistic Measure of Anthropomorphismcited by 27
- 2024MrT5: Dynamic Token Merging for Efficient Byte-level Language Modelscited by 27
- 2025Multi-turn Evaluation of Anthropomorphic Behaviours in Large Language Modelscited by 25
- 2024From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust?cited by 18
- 2021The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identitycited by 16
- 2024Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliancecited by 15
- 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 14
- 2024The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithmscited by 13
- 2022Robustness of Demonstration-based Learning Under Limited Data Scenariocited by 11
- 2024Characterizing and modeling harms from interactions with design patterns in AI interfacescited by 11
- 2024Instruction Following without Instruction Tuningcited by 11
- 2025From tools to thieves: Measuring and understanding public perceptions of AI through crowdsourced metaphorscited by 9
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2025: 7; 2026: 6.
13 papers.
Sort:
By area:
- 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 115
- 2025Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!cited by 36
- 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 35
- 2025A validity-guided workflow for robust large language model research in psychologycited by 8
- 2026Human Cognition in Machines: A Unified Perspective of World Modelscited by 5
- 2026Verbalizing LLMs' assumptions to explain and control sycophancycited by 3
- 2025Not Minds, but Signs: Reframing LLMs through Semioticscited by 2
- 2025Contemplative Artificial Intelligencecited by 1
- 2026Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deploymentscited by 1
- 2026Beliefs and Behavior in Language Modelscited by 1
- 2025Readability ne Learnability: Rethinking the Role of Simplicity in Training Small Language Modelscited by 0
- 2026Complacent, Not Sycophantic: Reframing Large Language Models and Designing AI Literacy for Complacent Machinescited by 0
- 2026Anthropomorphism in the age of Large Language Models: An overview of potential risks and mitigations29 Sep 2026
- 2026Beliefs and Behavior in Language Models7 Sep 2026
- 2026Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments14 Jun 2026
- 2026Complacent, Not Sycophantic: Reframing Large Language Models and Designing AI Literacy for Complacent Machines14 May 2026
- 2026Human Cognition in Machines: A Unified Perspective of World Models17 Apr 2026
- 2026Verbalizing LLMs' assumptions to explain and control sycophancy3 Apr 2026
- 2025Readability ne Learnability: Rethinking the Role of Simplicity in Training Small Language Models15 Oct 2025
- 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs3 Sep 2025
- 2025A validity-guided workflow for robust large language model research in psychology6 Jul 2025
- 2025Not Minds, but Signs: Reframing LLMs through Semiotics20 May 2025
- 2025ELEPHANT: Measuring and understanding social sycophancy in LLMs20 May 2025
- 2025Contemplative Artificial Intelligence21 Apr 2025
1 more
- 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 115
- 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMscited by 35
- 2025A validity-guided workflow for robust large language model research in psychologycited by 8
- 2026Verbalizing LLMs' assumptions to explain and control sycophancycited by 3
- 2025Not Minds, but Signs: Reframing LLMs through Semioticscited by 2
- 2025Contemplative Artificial Intelligencecited by 1
- 2026Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deploymentscited by 1
- 2026Beliefs and Behavior in Language Modelscited by 1
- 2026Complacent, Not Sycophantic: Reframing Large Language Models and Designing AI Literacy for Complacent Machinescited by 0
- 2026Anthropomorphism in the age of Large Language Models: An overview of potential risks and mitigationscited by 0
- 2026Anthropomorphism in the age of Large Language Models: An overview of potential risks and mitigations29 Sep 2026
- 2026Beliefs and Behavior in Language Models7 Sep 2026
- 2026Rethinking Scaffolding in LLM Tutors: The Interactional Mismatch Between Benchmarks and Real-World Deployments14 Jun 2026
- 2026Complacent, Not Sycophantic: Reframing Large Language Models and Designing AI Literacy for Complacent Machines14 May 2026
- 2026Verbalizing LLMs' assumptions to explain and control sycophancy3 Apr 2026
- 2025The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs3 Sep 2025
- 2025A validity-guided workflow for robust large language model research in psychology6 Jul 2025
- 2025Not Minds, but Signs: Reframing LLMs through Semiotics20 May 2025
- 2025ELEPHANT: Measuring and understanding social sycophancy in LLMs20 May 2025
- 2025Contemplative Artificial Intelligence21 Apr 2025
Similar, not cited
Close in wording, no citation either way
- 2025Humanizing Machines: Rethinking LLM Anthropomorphism Through a Multi-Level Framework of Design
- 2026A Scoping Review of the Ethical Perspectives on Anthropomorphising Large Language Model-Based Conversational Agents
- 2025Mind the Gap: The Divergence Between Human and LLM-Generated Tasks
- 2024Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina
- 2025Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications
- 2025Exploring Anthropomorphism in Conversational Agents for Environmental Sustainability
- 2026Understanding Large Language Models
- 2024LLM Theory of Mind and Alignment: Opportunities and Risks