# DeliChess: A Multi-party Dialogue Dataset for Deliberation in Chess Puzzle Solving

arXiv: https://arxiv.org/abs/2606.04987 · HTML: https://mlsanity.com/paper/2606.04987/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 46 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 7 of 7)

- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [A Repository of Conversational Datasets](https://mlsanity.com/paper/1904.06472/) (2019)
- [DeliData: A dataset for deliberation in multi-party problem solving](https://mlsanity.com/paper/2108.05271/) (2021)
- [I Beg to Differ: A study of constructive disagreement in online conversations](https://mlsanity.com/paper/2101.10917/) (2021)
- [Conversational Markers of Constructive Discussions](https://mlsanity.com/paper/1604.07407/) (2016)
- [Collaborative Evaluation of Deepfake Text with Deliberation-Enhancing Dialogue Systems](https://mlsanity.com/paper/2503.04945/) (2025)
- [Speed, Quality, and the Optimal Timing of Complex Decisions: Field Evidence](https://mlsanity.com/paper/2201.10808/) (2022)

## Abstract (arXiv)

Multi-party dialogue is a critical setting for studying collaborative reasoning and decision-making, yet existing datasets rarely focus on structured, reasoning-intensive tasks. We introduce DeliChess, a dataset of group deliberation dialogues in which participants collaboratively solve multiple-choice chess puzzles. Participants first answer independently, then engage in multi-party deliberation and revise their individual answers. The dataset comprises 107 dialogues with full transcripts, pre- and post-deliberation choices, and utterance-level annotations of communicative function, epistemic stance, and usefulness for supporting deliberation. Our analyses show that greater diversity in initial solution quality is associated with larger gains, while answer trajectories reveal how deliberation can recover, discover, or lose strong answers. Cases in which groups surpass every independent answer are associated with sustained reasoning and epistemic openness. We further propose a diagnostic action-selection task and find that the tested LLMs show only weak agreement with human-attested helpful actions. Together, our dataset provides a testbed for modelling group reasoning, dialogue dynamics, and conditions for effective deliberation.