mlsanityData to 7 Oct 2026

Enhance Reasoning for Large Language Models in the Game Werewolf

arXiv 2402.02330 ↗First posted 4 Feb 2024cs.AI cs.CLcited by 44

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

This paper presents an innovative framework that integrates Large Language Models (LLMs) with an external Thinker module to enhance the reasoning capabilities of LLM-based agents. Unlike augmenting LLMs with prompt engineering, Thinker directly harnesses knowledge from databases and employs various optimization techniques. The framework forms a reasoning hierarchy where LLMs handle intuitive System-1 tasks such as natural language processing, while the Thinker focuses on cognitive System-2 tasks that require complex logical analysis and domain-specific knowledge. Our framework is presented using a 9-player Werewolf game that demands dual-system reasoning. We introduce a communication protocol between LLMs and the Thinker, and train the Thinker using data from 18800 human sessions and reinforcement learning. Experiments demonstrate the framework's effectiveness in deductive reasoning, speech generation, and online game evaluation. Additionally, we fine-tune a 6B LLM to surpass GPT4 when integrated with the Thinker. This paper also contributes the largest dataset for social deduction games to date.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 49 of its 70 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

49 of 70 references.

References · 49

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2022Emergent Abilities of Large Language Modelscited by 2,483
37 more
  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  8. 2023PaLM 2 Technical Reportcited by 1,154
  9. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  10. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  11. 2023A Survey on Multimodal Large Language Modelscited by 898
  12. 2022Galactica: A Large Language Model for Sciencecited by 736
  13. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  14. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  15. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  16. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  17. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  18. 2018Behavioral Cloning from Observationcited by 482
  19. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  20. 2017Population Based Training of Neural Networkscited by 351
  21. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  22. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  23. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  24. 2022Exploring Length Generalization in Large Language Modelscited by 204
  25. 2023OpenAGI: When LLM Meets Domain Expertscited by 196
  26. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  27. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
  28. 2023ChatDB: Augmenting LLMs with Databases as Their Symbolic Memorycited by 136
  29. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 107
  30. 2023GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problemscited by 88
  31. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluationcited by 86
  32. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
  33. 2019Finding Friend and Foe in Multi-Agent Gamescited by 42
  34. 2023Playing the Werewolf game with artificial intelligence for language understandingcited by 16
  35. 2021RLupus: Cooperation through emergent communication in The Werewolf social deduction gamecited by 10
  36. 2022Hidden Agenda: a Social Deduction Game with Diverse Learned Equilibriacited by 6
  37. 2022A Novel Weighted Ensemble Learning Based Agent for the Werewolf Gamecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 16; 2026: 13.

44 papers.

Sort:

By area:

  1. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  2. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  3. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 97
  4. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 80
  5. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  6. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agentscited by 48
  7. 2024A Scalable Communication Protocol for Networks of Large Language Modelscited by 43
  8. 2024BattleAgentBench: A Benchmark for Evaluating Cooperation and Competition Capabilities of Language Models in Multi-Agent Systemscited by 26
  9. 2024InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Contextcited by 25
  10. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  11. 2024Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolfcited by 22
  12. 2024Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Searchcited by 16
32 more
  1. 2025Learning Strategic Language Agents in the Werewolf Game with Iterative Latent Space Policy Optimizationcited by 13
  2. 2024Helmsman of the Masses? Evaluate the Opinion Leadership of Large Language Models in the Werewolf Gamecited by 12
  3. 2025WebGames: Challenging General-Purpose Web-Browsing AI Agentscited by 12
  4. 2025MultiMind: Enhancing Werewolf Agents with Multimodal Reasoning and Theory of Mindcited by 11
  5. 2025Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Gamecited by 10
  6. 2024YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learningcited by 9
  7. 2025Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategiescited by 9
  8. 2025Bayesian Social Deduction with Graph-Informed Language Modelscited by 8
  9. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  10. 2025Deceive, Detect, and Disclose: Large Language Models Play Mini-Mafiacited by 7
  11. 2025MIRAGE: Exploring How Large Language Models Perform in Complex Social Interactive Environmentscited by 6
  12. 2025DVM: Towards Controllable LLM Agents in Social Deduction Gamescited by 6
  13. 2025Exploring Large Language Models for Word Games:Who is the Spy?cited by 6
  14. 2026Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Gamescited by 6
  15. 2025Empowering Economic Simulation for Massively Multiplayer Online Games through Generative Agent-Based Modelingcited by 3
  16. 2025The Stackelberg Speaker: Optimizing Persuasive Communication in Social Deduction Gamescited by 3
  17. 2026QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agentscited by 3
  18. 2025PRISON: Unmasking the Criminal Potential of Large Language Modelscited by 2
  19. 2025InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Stylescited by 2
  20. 2026Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiversecited by 2
  21. 2024GRATR: Zero-Shot Evidence Graph Retrieval-Augmented Trustworthiness Reasoningcited by 1
  22. 2025Verbal Werewolf: Engage Users with Verbalized Agentic Werewolf Game Frameworkcited by 1
  23. 2026MARO: Learning Stronger Reasoning from Social Interactioncited by 1
  24. 2026Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Keycited by 1
  25. 2025CSP4SDG: Constraint and Information-Theory Based Role Identification in Social Deduction Games with LLM-Enhanced Inferencecited by 0
  26. 2026SOM: Structured Opponent Modeling for LLM-based Agents via Structural Causal Modelcited by 0
  27. 2026When Clients Stop Following: A Cognitive Conceptualization Diagram-driven Framework for Strategic Counselingcited by 0
  28. 2026Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMscited by 0
  29. 2026CaM-Wolf: Causal-Aware Multimodal Agents for Social Deduction Gamescited by 0
  30. 2026Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Gamecited by 0
  31. 2026Persuasive and Compliant Tendencies Predict Group Decision-Making in Humans and Language Modelscited by 0
  32. 2026MARBO: Relational Belief Grounding for LLM Agents in Social Deduction Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025WereWolf-Plus: An Update of Werewolf Game setting Based on DSGBench
  2. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  3. 2026Clueing up LLMs with Tool-Augmented Deductive Reasoning
  4. 2025Game Reasoning Arena: A Framework and Benchmark for Assessing Reasoning Capabilities of Large Language Models via Game Play
  5. 2024An Implementation of Werewolf Agent That does not Truly Trust LLMs
  6. 2024LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models
  7. 2024GameArena: Evaluating LLM Reasoning through Live Computer Games
  8. 2024Towards Logically Sound Natural Language Reasoning with Logic-Enhanced Language Model Agents