Output Discipline Sets the Floor: A Falsification-Guided, Model-Agnostic Agent Protocol for Auditable Hypothesis-Driven Decision Making
On the map: LLM agents and tool use → Medical and scientific domain agents
Abstract (arXiv)
What binds large language models on auditable, hypothesis-driven decision tasks, where every candidate can be checked against the evidence? On the public TCMEval-SDT benchmark for Traditional Chinese Medicine syndrome differentiation (200 cases), the binding constraint is output discipline and reasoning structure, not model capability alone: two years of model scaling move zero-shot scores by only -0.03 to +0.08, while output-discipline failures (non-terminating reasoning, format non-compliance, large run-to-run spread) persist even in frontier models. We test this with a model-agnostic agent framework formalizing decision making as a five-stage closed loop, enforced by a candidate x evidence matrix falsification protocol: every candidate is judged against every evidence item, at least half of the candidates must be falsified before a conclusion is licensed, and survivors are emitted in a machine-verifiable single-line format. Across five base models, the effect is sign-conditional on zero-shot output discipline: it rescues models whose discipline fails (DeepSeek-V4-Flash: +0.267/+0.271, entirely termination rescue), reduces a high-variance model's run-to-run standard deviation from 0.218 to 0.007 (Kimi K3), and significantly costs already-disciplined models (-0.087 to -0.120 on budget-matched Qwen3-4B/32B). Ablations attribute the cost to the candidate-bounded format layer and show the effect is independent of injected knowledge content (protocol-knowledge decoupling). The protocol also transfers to misinformation-robust mathematical reasoning, detecting injected false premises at 89.8% recall with a 6.8% false-positive rate (MCC 0.54; a flag-everything baseline scores 0) and restoring accuracy to the no-misinformation ceiling. Where zero-shot output discipline is weak, the reasoning protocol sets the floor; base-model capability always sets the ceiling.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 5 of its 15 references that are in the map. First posted on arXiv 13 Oct 2026.
Builds on
5 of 15 references.
Closest ancestors · 1
Same direction, newest first
Other references · 4
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinkingcited by 16
- 2024Qibo: A Large Language Model for Traditional Chinese Medicinecited by 13
- 2024BianCang: A Traditional Chinese Medicine Large Language Modelcited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026RAudit: A Blind Auditing Protocol for Large Language Model Reasoning
- 2026FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games
- 2026ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence
- 2026Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs
- 2026Reasoning aligns language models to human cognition
- 2026When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMs
- 2026Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures
- 2026C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning