The Architecture of Errors: From Universal Impossibility to Patch-Local LLM Reliability
Not on the map. Why?
Abstract (arXiv)
Universal LLM reliability is not a finite-library problem: across all possible tasks, tools, schemas, knowledge sources, and evaluator expectations, new intervention-distinguishable failure modes can appear without bound, so no finite intervention dictionary can guarantee bounded residual error for every such mode. But deployed systems do not operate over the whole universe. They operate inside operationally bounded patches (legal review, medical RAG, code repair, customer-support agents, contract extraction) with recurring tasks, schemas, tools, and evaluator expectations. Within such patches, empirical evidence suggests failures are sparse, repetitive, and concentrated in a small recurring catalogue, so reliability becomes a local catalogue-discovery and intervention-coverage problem rather than an exponential token-length problem. We formalize this transition with two propositions and one corollary. Proposition 1 is the worst-case-mode-wise negative result: no finite intervention dictionary covers every distinguishable failure mode of an unbounded domain. Corollary 1 is the inverse-discovery implication: the logarithmic upper bound on mode discovery cannot accommodate linearly more distinct tail modes without exponentially more observed hard-failure events. Proposition 2 is the positive patch-local result: under log active-mode exposure and head-heavy coverage, a sufficient per-hard-decision intervention budget grows polylogarithmically in sequence length and becomes domain-constant once the patch catalogue saturates. The framework relocates rather than dissolves long-context difficulty: where the number of hard decisions itself grows with task length, reliability remains hard; the contribution is to identify the on-axis intervention rather than to make those regimes easy.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 58 of its 69 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
58 of 69 references.
References · 58
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Let's Verify Step by Stepcited by 3,598
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
- 2022Competition-Level Code Generation with AlphaCodecited by 1,427
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
- 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
- 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
38 more
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2022PAL: Program-aided Language Modelscited by 635
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
- 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
- 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
- 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 276
- 2025DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decompositioncited by 270
- 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
- 2024OmniParser for Pure Vision Based GUI Agentcited by 193
- 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
- 2025Measuring AI Ability to Complete Long Software Taskscited by 142
- 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 141
- 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
- 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
- 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 121
- 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
- 2024DebugBench: Evaluating Debugging Capability of Large Language Modelscited by 74
- 2024XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Modelscited by 73
- 2024What is Wrong with Perplexity for Long-context Language Modeling?cited by 68
- 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
- 2025How Do Large Language Monkeys Get Their Power (Laws)?cited by 40
- 2025Structured Uncertainty guided Clarification for LLM Agentscited by 34
- 2025SLOT: Structuring the Output of Large Language Modelscited by 18
- 2025Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Modelscited by 18
- 2025DINGO: Constrained Inference for Diffusion LLMscited by 15
- 2023Don't Fine-Tune, Decode: Syntax Error-Free Tool Use via Constrained Decodingcited by 13
- 2024Anchor-based Large Language Modelscited by 12
- 2025Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Modelscited by 11
- 2025From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculationscited by 11
- 2024AmbigNLG: Addressing Task Ambiguity in Instruction for NLGcited by 10
- 2025Error Classification of Large Language Models on Math Word Problems: A Dynamically Adaptive Frameworkcited by 10
- 2025Think, Prune, Train, Improve: Scaling Reasoning without Scaling Modelscited by 9
- 2024POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimizationcited by 8
- 2025A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QAcited by 8
50 most cited of 58 All 58 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Failure Modes in LLM Systems: A System-Level Taxonomy for Reliable AI Applications
- 2026Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes
- 2026When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime
- 2026Memory as Infrastructure: Reliability Engineering for Persistent Agent Memory in Months-Long LLM-Assisted Development
- 2026ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions
- 2026Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability
- 2026Confidently Wrong: Exception Chain Collapse in Frontier LLM Rule Evaluation
- 2025The Dual-State Architecture for Reliable LLM Agents