mlsanityData to 14 Oct 2026

When to Trust Context: Self-Reflective Debates for Context Reliability

arXiv 2506.06020 ↗First posted 6 Jun 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Large language models frequently encounter conflicts between their parametric knowledge and contextual input, often resulting in factual inconsistencies or hallucinations. We propose Self-Reflective Debate for Contextual Reliability (SR-DCR), a lightweight framework that integrates token-level self-confidence with an asymmetric multi-agent debate to adjudicate such conflicts. A critic, deprived of context, challenges a defender who argues from the given passage; a judge model evaluates the debate and determines the context's reliability. The final answer is selected by combining the verdict with model confidence. Experiments on the ClashEval benchmark demonstrate that SR-DCR consistently enhances robustness to misleading context while maintaining accuracy on trustworthy inputs, outperforming both classical debate and confidence-only baselines with minimal computational overhead. The code is available at https://github.com/smiles724/Self-Reflective-Debates.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

25 of 35 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  5. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  7. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  8. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
  9. 2018AI safety via debatecited by 400
  10. 2023Language Models can Solve Computer Taskscited by 372
  11. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  12. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
13 more
  1. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  2. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  3. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  4. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  5. 2022Large Language Models with Controllable Working Memorycited by 169
  6. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 113
  7. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
  8. 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Modelscited by 93
  9. 2023Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noisecited by 54
  10. 2023Mitigating Temporal Misalignment by Discarding Outdated Factscited by 21
  11. 2025When Persuasion Overrides Truth in Multi-Agent LLM Debates: Introducing a Confidence-Weighted Persuasion Override Rate (CW-POR)cited by 12
  12. 2025Exploring LLM Reasoning Through Controlled Prompt Variationscited by 10
  13. 2025Debate Helps Weak-to-Strong Generalizationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge
  2. 2026Black-Box Auditing of Epistemic Reliability in Multi-Agent Debate Distillation
  3. 2026Learning When to Trust via Selective Context Preference Optimization
  4. 2024Training Language Models to Win Debates with Self-Play Improves Judge Accuracy
  5. 2024To Trust or Not to Trust? Enhancing Large Language Models' Situated Faithfulness to External Contexts
  6. 2026Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
  7. 2025Enhancing Multi-Agent Debate System Performance via Confidence Expression
  8. 2026AI Safety via Debate is Compromised by Cognitive Biases