# To Mask or to Mirror: Human-AI Alignment in Collective Reasoning

arXiv: https://arxiv.org/abs/2510.01924 · HTML: https://mlsanity.com/paper/2510.01924/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 27 of its 67 references that are in the map.
- First posted on arXiv 2 Oct 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 27 of 27)

- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- [Simulating Opinion Dynamics with Networks of LLM-based Agents](https://mlsanity.com/paper/2311.09618/) (2023)
- [Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia](https://mlsanity.com/paper/2312.03664/) (2023)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [AgentReview: Exploring Peer Review Dynamics with LLM Agents](https://mlsanity.com/paper/2406.12708/) (2024)
- [Evaluating Large Language Model Biases in Persona-Steered Generation](https://mlsanity.com/paper/2405.20253/) (2024)
- [Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions](https://mlsanity.com/paper/2502.16761/) (2025)
- [When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour](https://mlsanity.com/paper/2311.09410/) (2023)
- [CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting](https://mlsanity.com/paper/2404.10199/) (2024)
- [Do LLM Agents Exhibit Social Behavior?](https://mlsanity.com/paper/2312.15198/) (2023)
- [Stereotype or Personalization? User Identity Biases Chatbot Recommendations](https://mlsanity.com/paper/2410.05613/) (2024)
- [The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents](https://mlsanity.com/paper/2311.09665/) (2023)
- [A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models](https://mlsanity.com/paper/2311.00445/) (2023)
- [Language Agents as Digital Representatives in Collective Decision-Making](https://mlsanity.com/paper/2502.09369/) (2025)
- [The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making](https://mlsanity.com/paper/2410.07304/) (2024)
- [Strategic Tradeoffs Between Humans and AI in Multi-Agent Bargaining](https://mlsanity.com/paper/2509.09071/) (2025)
- [Investigating LLMs as Voting Assistants via Contextual Augmentation: A Case Study on the European Parliament Elections 2024](https://mlsanity.com/paper/2407.08495/) (2024)
- [A Study of Nationality Bias in Names and Perplexity using Off-the-Shelf Affect-related Tweet Classifiers](https://mlsanity.com/paper/2407.01834/) (2024)
- [Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5?](https://mlsanity.com/paper/2311.07587/) (2023)
- [Evaluating LLMs for Gender Disparities in Notable Persons](https://mlsanity.com/paper/2403.09148/) (2024)
- [Assessing Group-level Gender Bias in Professional Evaluations: The Case of Medical Student End-of-Shift Feedback](https://mlsanity.com/paper/2206.00234/) (2022)

## Built on by (most cited first, 5 of 5)

- [Does Socialization Emerge in AI Agent Society? A Case Study of Moltbook](https://mlsanity.com/paper/2602.14299/) (2026)
- [Identity, Cooperation and Framing Effects within Groups of Real and Simulated Humans](https://mlsanity.com/paper/2601.16355/) (2026)
- [Discovering Differences in Strategic Behavior Between Humans and LLMs](https://mlsanity.com/paper/2602.10324/) (2026)
- [Memory as Metabolism: A Design for Companion Knowledge Systems](https://mlsanity.com/paper/2604.12034/) (2026)
- [Polite but Misaligned: Evaluating LLM Politeness Judgments Against Human Pragmatic Norms](https://mlsanity.com/paper/2609.29001/) (2026)

## Abstract (arXiv)

As large language models (LLMs) are increasingly used to model and augment collective decision-making, it is critical to examine their alignment with human social reasoning. We present an empirical framework for assessing collective alignment, in contrast to prior work on the individual level. Using the Lost at Sea social psychology task, we conduct a large-scale online experiment (N=748), randomly assigning groups to leader elections with either visible demographic attributes (e.g. name, gender) or pseudonymous aliases. We then simulate matched LLM groups conditioned on the human data, benchmarking Gemini 2.5, GPT 4.1, Claude Haiku 3.5, and Gemma 3. LLM behaviors diverge: some mirror human biases; others mask these biases and attempt to compensate for them. We empirically demonstrate that human-AI alignment in collective reasoning depends on context, cues, and model-specific inductive biases. Understanding how LLMs align with collective human behavior is critical to advancing socially-aligned AI, and demands dynamic benchmarks that capture the complexities of collective reasoning.