# Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies

arXiv: https://arxiv.org/abs/2510.11389 · HTML: https://mlsanity.com/paper/2510.11389/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Builds on (most cited first, 16 of 16)

- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf](https://mlsanity.com/paper/2309.04658/) (2023)
- [Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game](https://mlsanity.com/paper/2310.18940/) (2023)
- [Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.10708/) (2025)
- [Enhance Reasoning for Large Language Models in the Game Werewolf](https://mlsanity.com/paper/2402.02330/) (2024)
- [Werewolf Arena: A Case Study in LLM Evaluation via Social Deduction](https://mlsanity.com/paper/2407.13943/) (2024)
- [AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game](https://mlsanity.com/paper/2407.16521/) (2024)
- [The Stepwise Deception: Simulating the Evolution from True News to Fake News with LLM Agents](https://mlsanity.com/paper/2410.19064/) (2024)
- [CALYPSO: LLMs as Dungeon Masters' Assistants](https://mlsanity.com/paper/2308.07540/) (2023)
- [Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models](https://mlsanity.com/paper/2505.15406/) (2025)
- [Playing the Werewolf game with artificial intelligence for language understanding](https://mlsanity.com/paper/2302.10646/) (2023)
- [Helmsman of the Masses? Evaluate the Opinion Leadership of Large Language Models in the Werewolf Game](https://mlsanity.com/paper/2404.01602/) (2024)
- [Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models](https://mlsanity.com/paper/2311.05720/) (2023)
- [RLupus: Cooperation through emergent communication in The Werewolf social deduction game](https://mlsanity.com/paper/2106.05018/) (2021)
- [Hidden Agenda: a Social Deduction Game with Diverse Learned Equilibria](https://mlsanity.com/paper/2201.01816/) (2022)

## Built on by (most cited first, 10 of 10)

- [MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs](https://mlsanity.com/paper/2605.29512/) (2026)
- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)
- [QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents](https://mlsanity.com/paper/2605.27068/) (2026)
- [Self-Awareness before Action: Mitigating Logical Inertia via Proactive Cognitive Awareness](https://mlsanity.com/paper/2604.20413/) (2026)
- [Multicultural Spyfall: Assessing LLMs through Dynamic Multilingual Social Deduction Game](https://mlsanity.com/paper/2601.09017/) (2026)
- [Cooperate to Compete: Strategic Coordination in Multi-Agent Conquest](https://mlsanity.com/paper/2604.25088/) (2026)
- [Avalon-ToM-Bench: Evaluating Fine-Grained Theory of Mind via Asymmetric Game Mechanics](https://mlsanity.com/paper/2608.09638/) (2026)
- [Don' t Box Me In: Dynamic Cultural Adaptation and Cognitive Tracking for Social Understanding](https://mlsanity.com/paper/2608.22411/) (2026)
- [EVAR: Evidence-Validated Hypothesis Admission for Budget-Aware Narrative Reasoning](https://mlsanity.com/paper/2608.29835/) (2026)
- [Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation](https://mlsanity.com/paper/2610.01670/) (2026)

## Abstract (arXiv)

Social deduction games like Werewolf combine language, reasoning, and strategy, providing a testbed for studying natural language and social intelligence. However, most studies reduce the game to LLM-based self-play, yielding templated utterances and anecdotal cases that overlook the richness of social gameplay. Evaluation further relies on coarse metrics such as survival time or subjective scoring due to the lack of quality reference data. To address these gaps, we curate a high-quality, human-verified multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants. Based on this dataset, we propose a novel strategy-alignment evaluation that leverages the winning faction's strategies as ground truth in two stages: 1) Speech evaluation, formulated as multiple-choice-style tasks that assess whether the model can adopt appropriate stances across five dimensions of social ability; and 2) Decision evaluation, which assesses the model's voting choices and opponent-role inferences. This framework enables a fine-grained evaluation of models' linguistic and reasoning capabilities, while capturing their ability to generate strategically coherent gameplay. Our experiments show that state-of-the-art LLMs show diverse performance, with roughly half remain below 0.50, revealing clear gaps in deception and counterfactual reasoning. We hope our dataset further inspires research on language, reasoning, and strategy in multi-agent interaction.