# A Theory of Post-hoc Debate Judgement

arXiv: https://arxiv.org/abs/2608.19002 · HTML: https://mlsanity.com/paper/2608.19002/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 62 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 24 of 24)

- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://mlsanity.com/paper/2306.02561/) (2023)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023)
- [Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View](https://mlsanity.com/paper/2310.02124/) (2023)
- [Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication](https://mlsanity.com/paper/2312.01823/) (2023)
- [Argumentative XAI: A Survey](https://mlsanity.com/paper/2105.11266/) (2021)
- [Scalable AI Safety via Doubly-Efficient Debate](https://mlsanity.com/paper/2311.14125/) (2023)
- [Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate](https://mlsanity.com/paper/2402.07401/) (2024)
- [Interpreting Neural Networks as Gradual Argumentation Frameworks (Including Proof Appendix)](https://mlsanity.com/paper/2012.05738/) (2020)
- [Argumentative Large Language Models for Explainable and Contestable Claim Verification](https://mlsanity.com/paper/2405.02079/) (2024)
- [Interactive Explanations by Conflict Resolution via Argumentative Exchanges](https://mlsanity.com/paper/2303.15022/) (2023)
- [Multi-Agent Debate for LLM Judges with Adaptive Stability Detection](https://mlsanity.com/paper/2510.12697/) (2025)
- [Multiple LLM Agents Debate for Equitable Cultural Alignment](https://mlsanity.com/paper/2505.24671/) (2025)
- [Enhancing Multi-Agent Debate System Performance via Confidence Expression](https://mlsanity.com/paper/2509.14034/) (2025)
- [SpArX: Sparse Argumentative Explanations for Neural Networks [Technical Report]](https://mlsanity.com/paper/2301.09559/) (2023)
- [Demystifying Multi-Agent Debate: The Role of Confidence and Diversity](https://mlsanity.com/paper/2601.19921/) (2026)
- [Automatic Debate Evaluation with Argumentation Semantics and Natural Language Argument Graph Networks](https://mlsanity.com/paper/2203.14647/) (2022)
- [On the Discussion of Large Language Models: Symmetry of Agents and Interplay with Prompts](https://mlsanity.com/paper/2311.07076/) (2023)
- [Can LLMs Judge Debates? Evaluating Non-Linear Reasoning via Argumentation Theory Semantics](https://mlsanity.com/paper/2509.15739/) (2025)
- [Free Argumentative Exchanges for Explaining Image Classifiers](https://mlsanity.com/paper/2502.12995/) (2025)
- [Neurosymbolic Learning for Inference-Time Argumentation](https://mlsanity.com/paper/2605.20098/) (2026)

## Abstract (arXiv)

Debates have recently emerged as a useful methodology for agentic AI to improve performance as well as to aid explainability and user engagement. For example, LLM-empowered agents may debate internally (with themselves) and/or externally (with other agents). In many settings where debates are used, debates' outcomes and resulting outputs are determined post-hoc by external judges, often LLMs. In this paper we develop and test a novel theory of debate judgement applicable to all settings where agents engage in debates by providing pros and cons for their opinions therein. Specifically, we identify a number of formal properties that debate judgement may be required to satisfy in general, as concerns reproducibility, robustness, groundedness and explainability. Then, we explore their satisfaction formally and/or experimentally, for claim verification settings, for two specific alternative debate judgement methods: variants of the LLMs as a judge idea and formal semantics drawn from computational argumentation. We show that the two methods give similar accuracy performances but the former may lack formal guarantees that the latter brings. Overall, our study indicates argumentation semantics as an ideal candidate for principled judges in debate-driven AI.