# Werewolf Arena: A Case Study in LLM Evaluation via Social Deduction

arXiv: https://arxiv.org/abs/2407.13943 · HTML: https://mlsanity.com/paper/2407.13943/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://mlsanity.com/paper/2206.10498/) (2022)
- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf](https://mlsanity.com/paper/2309.04658/) (2023)
- [Large Language Models as Commonsense Knowledge for Large-Scale Task Planning](https://mlsanity.com/paper/2305.14078/) (2023)
- [Multi-agent Reinforcement Learning in Sequential Social Dilemmas](https://mlsanity.com/paper/1702.03037/) (2017)
- [Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game](https://mlsanity.com/paper/2310.18940/) (2023)
- [Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia](https://mlsanity.com/paper/2312.03664/) (2023)
- [Learning to Speak and Act in a Fantasy Text Adventure Game](https://mlsanity.com/paper/1903.03094/) (2019)
- [Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning](https://mlsanity.com/paper/1812.01628/) (2018)
- [MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding](https://mlsanity.com/paper/2106.01541/) (2021)
- [Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolf](https://mlsanity.com/paper/2405.19946/) (2024)
- [Simulating Human Strategic Behavior: Comparing Single and Multi-agent LLMs](https://mlsanity.com/paper/2402.08189/) (2024)
- [Playing the Werewolf game with artificial intelligence for language understanding](https://mlsanity.com/paper/2302.10646/) (2023)
- [Putting the Con in Context: Identifying Deceptive Actors in the Game of Mafia](https://mlsanity.com/paper/2207.02253/) (2022)
- [Is ChatGPT a Good Multi-Party Conversation Solver?](https://mlsanity.com/paper/2310.16301/) (2023)
- [Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games](https://mlsanity.com/paper/2212.08279/) (2022)

## Built on by (most cited first, 30 of 45)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [PointArena: Probing Multimodal Grounding Through Language-Guided Pointing](https://mlsanity.com/paper/2505.09990/) (2025)
- [A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios](https://mlsanity.com/paper/2412.03920/) (2024)
- [DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environments](https://mlsanity.com/paper/2503.06047/) (2025)
- [Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search](https://mlsanity.com/paper/2408.10635/) (2024)
- [Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontiers](https://mlsanity.com/paper/2502.09053/) (2025)
- [Learning Strategic Language Agents in the Werewolf Game with Iterative Latent Space Policy Optimization](https://mlsanity.com/paper/2502.04686/) (2025)
- [WOLF: Werewolf-based Observations for LLM Deception and Falsehoods](https://mlsanity.com/paper/2512.09187/) (2025)
- [SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments](https://mlsanity.com/paper/2505.23713/) (2025)
- [Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game](https://mlsanity.com/paper/2501.14225/) (2025)
- [Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies](https://mlsanity.com/paper/2510.11389/) (2025)
- [The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs](https://mlsanity.com/paper/2602.11583/) (2026)
- [Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafia](https://mlsanity.com/paper/2509.23023/) (2025)
- [M3-BENCH: Process-Aware Evaluation of LLM Agents' Social Behaviors in Mixed-Motive Games](https://mlsanity.com/paper/2601.08462/) (2026)
- [Who Speaks Next? Multi-party AI Discussion Leveraging the Systematics of Turn-taking in Murder Mystery Games](https://mlsanity.com/paper/2412.04937/) (2024)
- [One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligence](https://mlsanity.com/paper/2602.03109/) (2026)
- [TextAtari: 100K Frames Game Playing with Language Agents](https://mlsanity.com/paper/2506.04098/) (2025)
- [MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs](https://mlsanity.com/paper/2605.29512/) (2026)
- [PokerSkill: LLMs Can Play Expert-Level Poker without Training or Solvers](https://mlsanity.com/paper/2605.30094/) (2026)
- [PuzzleJAX: A Benchmark for Reasoning and Learning](https://mlsanity.com/paper/2508.16821/) (2025)
- [Evaluating Language Models' Evaluations of Games](https://mlsanity.com/paper/2510.10930/) (2025)
- [TongSIM: A General Platform for Simulating Intelligent Machines](https://mlsanity.com/paper/2512.20206/) (2025)
- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)
- [QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents](https://mlsanity.com/paper/2605.27068/) (2026)
- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Markets](https://mlsanity.com/paper/2505.20120/) (2025)
- [Strategy Adaptation in Large Language Model Werewolf Agents](https://mlsanity.com/paper/2507.12732/) (2025)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [Evaluating from Benign to Dynamic Adversarial: A Squid Game for Large Language Models](https://mlsanity.com/paper/2511.10691/) (2025)
- [Hidden in Plain Text: Measuring LLM Deception Quality Against Human Baselines Using Social Deduction Games](https://mlsanity.com/paper/2601.13709/) (2026)

## Abstract (arXiv)

This paper introduces Werewolf Arena, a novel framework for evaluating large language models (LLMs) through the lens of the classic social deduction game, Werewolf. In Werewolf Arena, LLMs compete against each other, navigating the game's complex dynamics of deception, deduction, and persuasion. The framework introduces a dynamic turn-taking system based on bidding, mirroring real-world discussions where individuals strategically choose when to speak. We demonstrate the framework's utility through an arena-style tournament featuring Gemini and GPT models. Our results reveal distinct strengths and weaknesses in the models' strategic reasoning and communication. These findings highlight Werewolf Arena's potential as a challenging and scalable LLM benchmark.