# Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games

arXiv: https://arxiv.org/abs/2312.00746 · HTML: https://mlsanity.com/paper/2312.00746/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Placed in

- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 16 of 16)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://mlsanity.com/paper/2106.14405/) (2021)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Character-LLM: A Trainable Agent for Role-Playing](https://mlsanity.com/paper/2310.10158/) (2023)
- [TextWorld: A Learning Environment for Text-based Games](https://mlsanity.com/paper/1806.11532/) (2018)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf](https://mlsanity.com/paper/2309.04658/) (2023)
- [MineRL: A Large-Scale Dataset of Minecraft Demonstrations](https://mlsanity.com/paper/1907.13440/) (2019)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [Combining Deep Reinforcement Learning and Search for Imperfect-Information Games](https://mlsanity.com/paper/2007.13544/) (2020)
- [Towards Playing Full MOBA Games with Deep Reinforcement Learning](https://mlsanity.com/paper/2011.12692/) (2020)

## Built on by (most cited first, 30 of 30)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models](https://mlsanity.com/paper/2404.01230/) (2024)
- [From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents](https://mlsanity.com/paper/2412.03563/) (2024)
- [AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game](https://mlsanity.com/paper/2407.16521/) (2024)
- [A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios](https://mlsanity.com/paper/2412.03920/) (2024)
- [A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges](https://mlsanity.com/paper/2403.10249/) (2024)
- [Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontiers](https://mlsanity.com/paper/2502.09053/) (2025)
- [Defending Against Social Engineering Attacks in the Age of LLMs](https://mlsanity.com/paper/2406.12263/) (2024)
- [Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detection](https://mlsanity.com/paper/2503.15552/) (2025)
- [PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Games](https://mlsanity.com/paper/2404.17662/) (2024)
- [Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game](https://mlsanity.com/paper/2501.14225/) (2025)
- [The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs](https://mlsanity.com/paper/2602.11583/) (2026)
- [AgentGroupChat: An Interactive Group Chat Simulacra For Better Eliciting Emergent Behavior](https://mlsanity.com/paper/2403.13433/) (2024)
- [MIRAGE: Exploring How Large Language Models Perform in Complex Social Interactive Environments](https://mlsanity.com/paper/2501.01652/) (2025)
- [HARBOR: Exploring Persona Dynamics in Multi-Agent Competition](https://mlsanity.com/paper/2502.12149/) (2025)
- [AI Agent Behavioral Science](https://mlsanity.com/paper/2506.06366/) (2025)
- [Parallelized Planning-Acting for Efficient LLM-based Multi-Agent Systems in Minecraft](https://mlsanity.com/paper/2503.03505/) (2025)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)
- [When Disagreements Elicit Robustness: Investigating Self-Repair Capabilities under LLM Multi-Agent Disagreements](https://mlsanity.com/paper/2502.15153/) (2025)
- [PRISON: Unmasking the Criminal Potential of Large Language Models](https://mlsanity.com/paper/2506.16150/) (2025)
- [BEDA: Belief Estimation as Probabilistic Constraints for Performing Strategic Dialogue Acts](https://mlsanity.com/paper/2512.24885/) (2025)
- [Is Your LLM Really Mastering the Concept? A Multi-Agent Benchmark](https://mlsanity.com/paper/2505.17512/) (2025)
- [MARO: Learning Stronger Reasoning from Social Interaction](https://mlsanity.com/paper/2601.12323/) (2026)
- [Self-Awareness before Action: Mitigating Logical Inertia via Proactive Cognitive Awareness](https://mlsanity.com/paper/2604.20413/) (2026)
- [IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environment](https://mlsanity.com/paper/2506.12331/) (2025)
- [Beneath the Surface: Investigating LLMs' Capabilities for Communicating with Subtext](https://mlsanity.com/paper/2604.05273/) (2026)
- [The Cost of Knowing: A Resource-Aware Protocol for Benchmarking Hallucination Beyond Static Leaderboards](https://mlsanity.com/paper/2607.24063/) (2026)
- [EVAR: Evidence-Validated Hypothesis Admission for Budget-Aware Narrative Reasoning](https://mlsanity.com/paper/2608.29835/) (2026)
- [Do LLMs Understand Sequential Structure? A Controlled Study of Inference and Generation](https://mlsanity.com/paper/2610.04977/) (2026)

## Abstract (arXiv)

In this study, we explore the application of Large Language Models (LLMs) in textitJubensha, a Chinese detective role-playing game and a novel area in Artificial Intelligence (AI) driven gaming. We introduce the first dataset specifically for Jubensha, including character scripts and game rules, to foster AI agent development in this complex narrative environment. Our work also presents a unique multi-agent interaction framework using LLMs, allowing AI agents to autonomously engage in this game. To evaluate the gaming performance of these AI agents, we developed novel methods measuring their mastery of case information and reasoning skills. Furthermore, we incorporated the latest advancements in in-context learning to improve the agents' performance in information gathering, murderer identification, and logical reasoning. The experimental results validate the effectiveness of our proposed methods. This work aims to offer a novel perspective on understanding LLM capabilities and establish a new benchmark for evaluating large language model-based agents.