mlsanityData to 14 Oct 2026

Human-LLM Deliberation as Interactive Proof: Conditions for Verifiability Without Transparency

arXiv 2609.24895 ↗First posted 21 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

When an LLM supplies an argument a user could not produce, how can the user decide whether to accept its claim? Inspired by interactive proofs, we pair a computationally unbounded LLM prover with a resource-bounded human verifier. The verifier checks requested details without seeing the LLM's internals; passed checks accumulate evidence toward an acceptance threshold. Given task-supplied bounds on false passes and human checking errors, we prove anytime-valid soundness against adaptive provers: the probability of ever accepting a false claim is at most a chosen error level. For a true claim, acceptance within the budget has probability at least the product of per-round probabilities that the LLM responds adequately and the human checks correctly, provided the threshold is reachable in time. Each additional check guards against false claims but gives a true claim another chance to fail. Which review lengths meet both error targets depends on the verifier's effort, cognitive load, expertise, and fatigue. With equal resources, we identify conditions under which the bounds certify a review by local checks but not a one-shot check. In LLM-only simulations on GPQA Diamond and Humanity's Last Exam, whether answers accepted without revision were more accurate varied by task and model pair.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

42 of 59 references.

References · 42

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  7. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  8. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  9. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  10. 2017Selective Classification for Deep Neural Networkscited by 852
  11. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  12. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
30 more
  1. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  2. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 427
  3. 2018AI safety via debatecited by 400
  4. 2022Self-critiquing models for assisting human evaluatorscited by 380
  5. 2023Explainability for Large Language Models: A Surveycited by 333
  6. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  7. 2022Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofscited by 291
  8. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  9. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  10. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 225
  11. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  12. 2022Game-theoretic statistics and safe anytime-valid inferencecited by 155
  13. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  14. 2018Supervising strong learners by amplifying weak expertscited by 145
  15. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 105
  16. 2024On scalable oversight with weak LLMs judging strong LLMscited by 99
  17. 2024Prover-Verifier Games improve legibility of LLM outputscited by 85
  18. 2023Scalable AI Safety via Doubly-Efficient Debatecited by 57
  19. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  20. 2023In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Makingcited by 50
  21. 2023Debate Helps Supervise Unreliable Expertscited by 43
  22. 2024Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Makingcited by 38
  23. 2021Learning to Give Checkable Answers with Prover-Verifier Gamescited by 32
  24. 2025E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testingcited by 17
  25. 2023Why is plausibility surprisingly problematic as an XAI criterion?cited by 9
  26. 2024Neural Interactive Proofscited by 7
  27. 2026Trust but Verify: Prover-Verifier Deliberation for Selective LLM Predictioncited by 4
  28. 2026When Should an AI Workflow Release? Always-Valid Inference for Black-Box Generate-Verify Systemscited by 3
  29. 2026How to Avoid Debate: Scalable AI Safety via Doubly-Efficient Interactive Proofscited by 2
  30. 2026The Impossibility of Eliciting Latent Knowledgecited by 1

Built on by

Not cited yet.