mlsanityData to 14 Oct 2026

Exploring Large Language Models for Word Games:Who is the Spy?

arXiv 2503.15235 ↗First posted 19 Mar 2025cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Word games hold significant research value for natural language processing (NLP), game theory, and related fields due to their rule-based and situational nature. This study explores how large language models (LLMs) can be effectively involved in word games and proposes a training-free framework. "Shei Shi Wo Di" or "Who is the Spy" in English, is a classic word game. Using this game as an example, we introduce a Chain-of-Thought (CoT)-based scheduling framework to enable LLMs to achieve excellent performance in tasks such as inferring role words and disguising their identities. We evaluate the framework's performance based on game success rates and the accuracy of the LLM agents' analytical results. Experimental results affirm the framework's effectiveness, demonstrating notable improvements in LLM performance across multiple datasets. This work highlights the potential of LLMs in mastering situational reasoning and social interactions within structured game environments. Our code is publicly available at https://github.com/ct-wei/Who-is-The-Spy.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 24 of its 25 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

24 of 25 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  8. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  9. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  10. 2022Large Language Models Are Reasoning Teacherscited by 329
  11. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 241
  12. 2024Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?cited by 159
12 more
  1. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 152
  2. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
  3. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 135
  4. 2023The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuningcited by 108
  5. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 96
  6. 2023ChessGPT: Bridging Policy Learning and Language Modelingcited by 75
  7. 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplaycited by 63
  8. 2024Enhance Reasoning for Large Language Models in the Game Werewolfcited by 46
  9. 2020The Chess Transformer: Mastering Play using Generative Language Modelscited by 37
  10. 2024Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Taskscited by 22
  11. 2024LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agentscited by 5
  12. 2024Making New Connections: LLMs as Puzzle Generators for The New York Times' Connections Word Gamecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environmentscited by 11
  2. 2025PRISON: Unmasking the Criminal Potential of Large Language Modelscited by 2
  3. 2025Is Your LLM Really Mastering the Concept? A Multi-Agent Benchmarkcited by 1
  4. 2026GIFT: Games as Informal Training for Generalizable LLMscited by 1
  5. 2025Can LLMs Play O uAn Quan Game? A Study of Multi-Step Planning and Decision Makingcited by 0
  6. 2025What to Ask Next? Probing the Imaginative Reasoning of LLMs with TurtleSoup Puzzlescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models
  2. 2026Evaluating Large Language Models in a Complex Hidden Role Game
  3. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  4. 2023Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games
  5. 2025Can LLMs effectively provide game-theoretic-based scenarios for cybersecurity?
  6. 2025Baba is LLM: Reasoning in a Game with Dynamic Rules
  7. 2024Evaluating Creativity and Deception in Large Language Models: A Simulation Framework for Multi-Agent Balderdash
  8. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios