# Counting Moves, Weighing Voices: Bayesian Dialectical Argumentation for Calibrated Multi-LLM Councils under Persistent Adversaries

arXiv: https://arxiv.org/abs/2610.02005 · HTML: https://mlsanity.com/paper/2610.02005/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 59 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Builds on (most cited first, 14 of 14)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Mixture-of-Agents Enhances Large Language Model Capabilities](https://mlsanity.com/paper/2406.04692/) (2024)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Debating with More Persuasive LLMs Leads to More Truthful Answers](https://mlsanity.com/paper/2402.06782/) (2024)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?](https://mlsanity.com/paper/2502.00674/) (2025)
- [Argumentative Large Language Models for Explainable and Contestable Claim Verification](https://mlsanity.com/paper/2405.02079/) (2024)
- [Free-MAD: Consensus-Free Multi-Agent Debate](https://mlsanity.com/paper/2509.11035/) (2025)
- [MAD-Spear: A Conformity-Driven Prompt Injection Attack on Multi-Agent Debate Systems](https://mlsanity.com/paper/2507.13038/) (2025)
- [Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate](https://mlsanity.com/paper/2606.00820/) (2026)
- [Can LLMs Judge Debates? Evaluating Non-Linear Reasoning via Argumentation Theory Semantics](https://mlsanity.com/paper/2509.15739/) (2025)
- [MArgE: Meshing Argumentative Evidence from Multiple Large Language Models for Justifiable Claim Verification](https://mlsanity.com/paper/2508.02584/) (2025)

## Abstract (arXiv)

A multi-LLM council lets several large language models (LLMs) deliberate on a question and return an answer together with a confidence estimate. As these systems become increasingly used for reasoning, that confidence should represent a calibrated probability of being correct, and the decision should remain robust when some agents are persistently unreliable. Existing council aggregation methods fail on both fronts: their confidence estimates measure decisiveness rather than correctness, and they cannot identify or discount persistently unreliable agents. We introduce Bayesian Dialectical Argumentation (BDA), which treats the council's typed moves---who proposed, challenged, or conceded which answer---as observations of a classical annotator model with per-agent reliabilities. This formulation recasts multi-agent deliberation as a reliability estimation problem, using the deliberation trace to infer agent reliability under persistent adversarial behavior. By weighting evidence according to inferred agent reliability, BDA yields calibrated posterior probabilities over candidate answers while allowing persistently unreliable agents to be inverted rather than merely outvoted. Across binary and multi-class benchmarks, BDA achieves the best calibration among zero-cost council aggregation methods, requiring no additional LLM calls, and improves robustness under persistent adversarial coalitions while remaining competitive in clean settings.