mlsanityData to 7 Oct 2026

Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games

arXiv 2312.00746 ↗First posted 1 Dec 2023cs.AIcited by 29

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

In this study, we explore the application of Large Language Models (LLMs) in textitJubensha, a Chinese detective role-playing game and a novel area in Artificial Intelligence (AI) driven gaming. We introduce the first dataset specifically for Jubensha, including character scripts and game rules, to foster AI agent development in this complex narrative environment. Our work also presents a unique multi-agent interaction framework using LLMs, allowing AI agents to autonomously engage in this game. To evaluate the gaming performance of these AI agents, we developed novel methods measuring their mastery of case information and reasoning skills. Furthermore, we incorporated the latest advancements in in-context learning to improve the agents' performance in information gathering, murderer identification, and logical reasoning. The experimental results validate the effectiveness of our proposed methods. This work aims to offer a novel perspective on understanding LLM capabilities and establish a new benchmark for evaluating large language model-based agents.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 16 of its 37 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

16 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge17 Jun

Other references · 15

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  5. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  7. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  8. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  9. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  10. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  11. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  12. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
3 more
  1. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  2. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 110
  3. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 14; 2026: 6.

29 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  3. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 97
  4. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  5. 2024AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Gamecited by 26
  6. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  7. 2024A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challengescited by 22
  8. 2024Defending Against Social Engineering Attacks in the Age of LLMscited by 15
  9. 2025Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontierscited by 14
  10. 2025Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detectioncited by 13
  11. 2024PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Gamescited by 10
  12. 2025Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Gamecited by 10
17 more
  1. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  2. 2024AgentGroupChat: An Interactive Group Chat Simulacra For Better Eliciting Emergent Behaviorcited by 6
  3. 2025MIRAGE: Exploring How Large Language Models Perform in Complex Social Interactive Environmentscited by 6
  4. 2025HARBOR: Exploring Persona Dynamics in Multi-Agent Competitioncited by 6
  5. 2025Parallelized Planning-Acting for Efficient LLM-based Multi-Agent Systems in Minecraftcited by 5
  6. 2025AI Agent Behavioral Sciencecited by 5
  7. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  8. 2025When Disagreements Elicit Robustness: Investigating Self-Repair Capabilities under LLM Multi-Agent Disagreementscited by 3
  9. 2025PRISON: Unmasking the Criminal Potential of Large Language Modelscited by 2
  10. 2025BEDA: Belief Estimation as Probabilistic Constraints for Performing Strategic Dialogue Actscited by 2
  11. 2025Is Your LLM Really Mastering the Concept? A Multi-Agent Benchmarkcited by 1
  12. 2026MARO: Learning Stronger Reasoning from Social Interactioncited by 1
  13. 2026Self-Awareness before Action: Mitigating Logical Inertia via Proactive Cognitive Awarenesscited by 1
  14. 2025IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environmentcited by 0
  15. 2026Beneath the Surface: Investigating LLMs' Capabilities for Communicating with Subtextcited by 0
  16. 2026The Cost of Knowing: A Resource-Aware Protocol for Benchmarking Hallucination Beyond Static Leaderboardscited by 0
  17. 2026EVAR: Evidence-Validated Hypothesis Admission for Budget-Aware Narrative Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games
  2. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  3. 2025Exploring Large Language Models for Word Games:Who is the Spy?
  4. 2024LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agents
  5. 2026Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games
  6. 2024Large Language Models as Agents in Two-Player Games
  7. 2024GAMEBoT: Transparent Assessment of LLM Reasoning in Games
  8. 2024LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environments