Auditing Belief-Conditioned LLM Agents in Hidden-Information Social Deduction Games
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
Evaluating LLM agents in hidden-information multi-agent settings is hard: final outcomes are high-variance and rarely reveal why an agent decided as it did. We study this in a 9-player Werewolf environment where agents act under strict, code-level information isolation, and we build an auditable framework that maintains an external belief state over hidden roles, logs belief updates and belief-action deviations as structured evidence, and supports a defensive offline improvement loop that reviews bad cases before any strategy change. Across 1,080 frozen games spanning belief-disabled, active-belief, kernel-ablation, camp-restricted, consumption-policy, and high-load arms, and including a seed-paired A0/A1 comparison, the active-belief condition is associated with substantially better good-side outcomes: in the 200-seed A0/A1 comparison the good-side win rate rises from 0.205 to 0.390 (paired McNemar chi2 = 16.4, p < 0.001), with fewer irreversible witch-poison errors. We do not, however, attribute this shift to belief content. Direct action-belief consistency is low (≈ 0.21), and giving belief only to the werewolves helps the good side more than giving it only to the good side, which argues against a simple holder-benefit account; we therefore report the effect as an association and treat its mechanism as unresolved. The contribution is the audit framework itself: it makes the effect measurable, exposes low direct action-belief consistency, rejects an unreliable forced-consumption intervention with evidence, and separates strategy effects from load confounds. We accordingly position external belief in high-noise hidden-information games primarily as an auditable cognitive baseline that also carries decision-relevant signal, turning opaque agent behavior into replayable evidence for safer, controlled iteration.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 20 of its 28 references that are in the map. First posted on arXiv 12 Jul 2026.
Builds on
20 of 28 references.
Closest ancestors · 4
Same direction, newest first
- 2026MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs28 May
- 2025Learning Strategic Language Agents in the Werewolf Game with Iterative Latent Space Policy Optimization7 Feb
- 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game29 Oct
- 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar
Other references · 16
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 233
- 2018Machine Theory of Mindcited by 226
- 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 114
- 2018Deep Counterfactual Regret Minimizationcited by 103
- 2021Off-Belief Learningcited by 75
- 2018Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 68
- 2020Agent Modelling under Partial Observability for Deep Reinforcement Learningcited by 57
4 more
- 2024Werewolf Arena: A Case Study in LLM Evaluation via Social Deductioncited by 45
- 2019Finding Friend and Foe in Multi-Agent Gamescited by 42
- 2025WOLF: Werewolf-based Observations for LLM Deception and Falsehoodscited by 13
- 2026Hidden in Plain Text: Measuring LLM Deception Quality Against Human Baselines Using Social Deduction Gamescited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MafiaScope: Non-Invasive, Time-Resolved Belief Probing for LLM Agents in Social Deduction Games
- 2026Confident at the moment of action: belief miscalibration in LLM play under hidden information
- 2026Do LLMs Trust the Accuser or the Accusation? Measuring Belief Shifts in Werewolf
- 2026Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systems
- 2026TriEx: A Game-based Tri-View Framework for Explaining Internal Reasoning in Multi-Agent LLMs
- 2026Don't Make the LLM Read the Graph: Make the Graph Think
- 2025DVM: Towards Controllable LLM Agents in Social Deduction Games
- 2026Differentiable Belief-based Opponent Shaping