mlsanityData to 14 Oct 2026

Evidence-State Reliability Under Controlled Degradation: Parser-Validity Divergence in a Multi-Stage LLM Pipeline

arXiv 2608.21559 ↗First posted 21 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-stage LLM pipelines can remain structurally valid even when evidence available to downstream stages becomes incomplete, compressed, or conflicting. This paper introduces and operationalizes Evidence-State Reliability (ESR), an evaluation layer concerned with whether intermediate evidence remains sufficiently complete, grounded, internally consistent, and usable for a stage's assigned function. ESR is evaluated separately from parser validity, which measures structural conformance. We evaluate the framework using GLM-5.2 on 60 sanitized base cases under four evidence conditions: clean, compressed-lossy, partial-dropout, and noisy-conflicting. Each condition was processed through decision, audit, and escalation stages. The design comprised 720 planned and ledgered calls, with 713 retained, sanitized execution rows. Across nine matched degraded-minus-clean condition-stage comparisons, all operational stage-success estimates were negative, and all 95% bootstrap intervals remained below zero. All nine parser-validity point estimates were positive, although the three partial-dropout intervals included zero. Among parser-valid degraded audit outputs, degradation detection was 1.0 in each degraded condition, while false-assurance rates remained non-zero; among parser-valid degraded escalation outputs, recovery was 0.0 in every degraded condition. The results show a bounded reliability-layer divergence in the evaluated pipeline: structural conformance can improve directionally while evidence-sensitive stage success deteriorates under the same controlled intervention. They also separate detection of degraded evidence from recovery. The conclusions are limited to the evaluated model configuration, pipeline design, selected sanitized cases, scoring procedure, and single scaled run.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 21 Aug 2026.

Builds on

16 of 23 references.

References · 16

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  3. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 418
  4. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 384
  5. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  6. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
  7. 2020Consistent Estimators for Learning to Defer to an Expertcited by 271
  8. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 265
  9. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  10. 2026From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaborationcited by 14
  11. 2026Hallucination Cascade: Analyzing Error Propagation in Multi-Agent LLM Systemscited by 12
  12. 2022Fact Checking with Insufficient Evidencecited by 11
4 more
  1. 2026SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generationcited by 8
  2. 2026The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Modelscited by 4
  3. 2026The Structured Output Benchmark: A Multi-Source Benchmark for Evaluating Structured Output Quality in Large Language Modelscited by 3
  4. 2026Evidence Sufficiency Under Delayed Ground Truth: Proxy Monitoring for Risk Decision Systemscited by 2

Built on by

Not cited yet.