# Probabilistic Consensus through Ensemble Validation: A Framework for LLM Reliability

arXiv: https://arxiv.org/abs/2411.06535 · HTML: https://mlsanity.com/paper/2411.06535/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 8 references that are in the map.
- First posted on arXiv 10 Nov 2024.

## Builds on (most cited first, 1 of 1)

- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)

## Built on by (most cited first, 9 of 9)

- [LLM-Powered AI Agent Systems and Their Applications in Industry](https://mlsanity.com/paper/2505.16120/) (2025)
- [Collective Reasoning Among LLMs: A Framework for Answer Validation Without Ground Truth](https://mlsanity.com/paper/2502.20758/) (2025)
- [Enhancing Answer Reliability Through Inter-Model Consensus of Large Language Models](https://mlsanity.com/paper/2411.16797/) (2024)
- [The Six Sigma Agent: Achieving Enterprise-Grade Reliability in LLM Systems Through Consensus-Driven Decomposed Execution](https://mlsanity.com/paper/2601.22290/) (2026)
- [If You Want Coherence, Orchestrate a Team of Rivals: Multi-Agent Models of Organizational Intelligence](https://mlsanity.com/paper/2601.14351/) (2026)
- [Evaluating LLM Safety Under Repeated Inference via Accelerated Prompt Stress Testing](https://mlsanity.com/paper/2602.11786/) (2026)
- [Evaluating Reliability Gaps in Large Language Model Safety via Repeated Prompt Sampling](https://mlsanity.com/paper/2604.09606/) (2026)
- [Metric assessment protocol in the context of answer fluctuation on MCQ tasks](https://mlsanity.com/paper/2507.15581/) (2025)
- [Position: Vibe Coding Needs Vibe Reasoning: Improving Vibe Coding with Formal Verification](https://mlsanity.com/paper/2511.00202/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have shown significant advances in text generation but often lack the reliability needed for autonomous deployment in high-stakes domains like healthcare, law, and finance. Existing approaches rely on external knowledge or human oversight, limiting scalability. We introduce a novel framework that repurposes ensemble methods for content validation through model consensus. In tests across 78 complex cases requiring factual accuracy and causal consistency, our framework improved precision from 73.1% to 93.9% with two models (95% CI: 83.5%-97.9%) and to 95.6% with three models (95% CI: 85.2%-98.8%). Statistical analysis indicates strong inter-model agreement (kappa > 0.76) while preserving sufficient independence to catch errors through disagreement. We outline a clear pathway to further enhance precision with additional validators and refinements. Although the current approach is constrained by multiple-choice format requirements and processing latency, it offers immediate value for enabling reliable autonomous AI systems in critical applications.