# Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models

arXiv: https://arxiv.org/abs/2311.05720 · HTML: https://mlsanity.com/paper/2311.05720/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 9 Nov 2023.

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [StructGPT: A General Framework for Large Language Model to Reason over Structured Data](https://mlsanity.com/paper/2305.09645/) (2023)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Deal or No Deal? End-to-End Learning for Negotiation Dialogues](https://mlsanity.com/paper/1706.05125/) (2017)
- [Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good](https://mlsanity.com/paper/1906.06725/) (2019)
- [A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity](https://mlsanity.com/paper/2305.13169/) (2023)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [Language-Conditioned Imitation Learning for Robot Manipulation Tasks](https://mlsanity.com/paper/2010.12083/) (2020)
- [Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions](https://mlsanity.com/paper/1602.01103/) (2016)
- [Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks](https://mlsanity.com/paper/1708.04299/) (2017)
- [Dialogue State Tracking with a Language Model using Schema-Driven Prompting](https://mlsanity.com/paper/2109.07506/) (2021)
- [CoLT5: Faster Long-Range Transformers with Conditional Computation](https://mlsanity.com/paper/2303.09752/) (2023)
- [Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning](https://mlsanity.com/paper/2210.05492/) (2022)
- [Finding Friend and Foe in Multi-Agent Games](https://mlsanity.com/paper/1906.02330/) (2019)
- [Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2008.06239/) (2020)
- [Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi](https://mlsanity.com/paper/2107.07630/) (2021)
- [Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning](https://mlsanity.com/paper/2112.03763/) (2021)
- [Towards Universal Dialogue State Tracking](https://mlsanity.com/paper/1810.09587/) (2018)
- [Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning](https://mlsanity.com/paper/1606.02560/) (2016)
- [How to Ask for a Favor: A Case Study on the Success of Altruistic Requests](https://mlsanity.com/paper/1405.3282/) (2014)
- [Visual Dialogue State Tracking for Question Generation](https://mlsanity.com/paper/1911.07928/) (2019)
- [Putting the Con in Context: Identifying Deceptive Actors in the Game of Mafia](https://mlsanity.com/paper/2207.02253/) (2022)
- [Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games](https://mlsanity.com/paper/2212.08279/) (2022)

## Built on by (most cited first, 12 of 12)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations](https://mlsanity.com/paper/2402.12348/) (2024)
- [How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments](https://mlsanity.com/paper/2403.11807/) (2024)
- [Pok'eChamp: an Expert-level Minimax Language Agent](https://mlsanity.com/paper/2503.04094/) (2025)
- [An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Making](https://mlsanity.com/paper/2410.15168/) (2024)
- [Beyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies](https://mlsanity.com/paper/2510.11389/) (2025)
- [The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs](https://mlsanity.com/paper/2602.11583/) (2026)
- [More Victories, Less Cooperation: Assessing Cicero's Diplomacy Play](https://mlsanity.com/paper/2406.04643/) (2024)
- [Bayesian Social Deduction with Graph-Informed Language Models](https://mlsanity.com/paper/2506.17788/) (2025)
- [Understanding Your Agent: Leveraging Large Language Models for Behavior Explanation](https://mlsanity.com/paper/2311.18062/) (2023)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game](https://mlsanity.com/paper/2607.28146/) (2026)

## Abstract (arXiv)

Deception and persuasion play a critical role in long-horizon dialogues between multiple parties, especially when the interests, goals, and motivations of the participants are not aligned. Such complex tasks pose challenges for current Large Language Models (LLM) as deception and persuasion can easily mislead them, especially in long-horizon multi-party dialogues. To this end, we explore the game of Avalon: The Resistance, a social deduction game in which players must determine each other's hidden identities to complete their team's objective. We introduce an online testbed and a dataset containing 20 carefully collected and labeled games among human players that exhibit long-horizon deception in a cooperative-competitive setting. We discuss the capabilities of LLMs to utilize deceptive long-horizon conversations between six human players to determine each player's goal and motivation. Particularly, we discuss the multimodal integration of the chat between the players and the game's state that grounds the conversation, providing further insights into the true player identities. We find that even current state-of-the-art LLMs do not reach human performance, making our dataset a compelling benchmark to investigate the decision-making and language-processing capabilities of LLMs. Our dataset and online testbed can be found at our project website: https://sstepput.github.io/Avalon-NLU/