mlsanityData to 14 Oct 2026

Council Mode: A Heterogeneous Multi-Agent Consensus Framework for Reducing LLM Hallucination and Bias

arXiv 2604.02923 ↗First posted 3 Apr 2026cs.CL cs.AIcited by 1

On the map: LLM agents and tool use → Medical and scientific domain agents

also in 1 more
Abstract (arXiv)

Long-form research questions often require answers that combine complementary details, reconcile conflicting estimates, and retain qualifications across several large language model (LLM) outputs. We present an inspectable LLM Council with two synthesis stages: an analyst organizes three independent candidate answers into a structured textual state, and a separately invoked writer composes the final response from the question and that state. We evaluate the workflow on 100 closed-book tasks from the Deep Research Accuracy, Completeness, and Objectivity (DRACO) benchmark, retaining 600 final answers, 100 analyst states, and 720 whole-answer judgments. Under the primary rubric judge, Council scores 70.98, exceeding GPT and Gemini by 6.18 and 8.34 points; its 0.47-point difference from Claude has a 95% paired interval of [-0.69,1.61]. In the four-arm fusion comparison, Council scores 1.87 points above direct fusion (95% paired interval [0.33,3.48]) and covers 51.1% of positive criteria covered by exactly one candidate, compared with 43.7% for direct fusion. On an exploratory 68-task subset with matching returned model labels, direct fusion scores higher. The full comparison characterizes workflows that differ in structure and generation budget; estimating the analyst stage's effect requires a controlled component study. Traced cases show how selected estimates, provenance qualifications, and selection decisions pass through the saved state into final answers.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 16 of its 18 references that are in the map. First posted on arXiv 3 Apr 2026.

Builds on

16 of 18 references.

References · 16

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  5. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  6. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 438
  7. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  8. 2023Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMscited by 114
  9. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?cited by 54
  10. 2025Correlated Errors in Large Language Modelscited by 45
  11. 2025LLMs Can Generate a Better Answer by Aggregating Their Own Responsescited by 27
  12. 2024Language Model Council: Democratically Benchmarking Foundation Models on Highly Subjective Taskscited by 18
4 more
  1. 2026DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivitycited by 14
  2. 2025Learning to Refine: Self-Refinement of Parallel Reasoning in LLMscited by 12
  3. 2025Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judgecited by 11
  4. 2025Progressive Document-level Text Simplification via Large Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Multi-Agent Consensus through Third-Party LLM Integration: Analyzing Uncertainty and Mitigating Hallucinations in Large Language Models
  2. 2024Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework
  3. 2026Hallucination Cascade: Analyzing Error Propagation in Multi-Agent LLM Systems
  4. 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
  5. 2026Collective Hallucination in Multi-Agent LLMs:Modeling and Defense
  6. 2026Reasoning Error from Known Fact: Step-Level Self-Consistency Group Relative Policy Optimization for LLM
  7. 2024LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
  8. 2025OnionEval: An Unified Evaluation of Fact-conflicting Hallucination for Small-Large Language Models