mlsanityData to 14 Oct 2026

Never the Number: Structural Abstention for AI Systems Whose Answers Are Consumed as Fact

arXiv 2608.13926 ↗First posted 14 Aug 2026cs.AI cs.CL cs.DBcited by 1

Not on the map. Why?

Abstract (arXiv)

Large language models have made natural language interfaces to databases (NLIDB) newly credible, but LLM text-to-SQL systems fail in a way that matters for deployment: a hallucinated column or a mis-aggregated total yields a fluent wrong answer, indistinguishable at the point of use from a right one. Where the consumer cannot inspect the generated query, as in enterprise AI deployments and operational dashboards, and increasingly where the consumer is a tool-using agent rather than a person, accuracy alone is insufficient: nothing marks which answers to distrust. This is a reliability problem before it is an accuracy problem. We propose an architectural pattern for such systems, a trusted kernel with a generative shell, resting on one invariant: a component that can fabricate may influence which question the system answers, never which value it returns. A generative shell interprets underspecified input and phrases replies; a deterministic kernel matches fully specified questions against a bounded set of answerable question shapes and compiles them to queries by deterministic execution. The two meet at a confirmation the user reads before any value is computed, and requests the kernel cannot express are declined rather than approximated. We call this structural abstention, and distinguish it from the statistical abstention of selective prediction and calibrated confidence: refusal here needs no confidence estimate, because unanswerable requests are unrepresentable. We specify the pattern implementation-independently, give a five-decision recipe and work it across three domains, extend the invariant from returned values to the actions of agentic systems, and report a two-year production case study alongside two generative alternatives, a fine-tuned parser and a tool-retrieval agent. We close against enterprise and reliability benchmarks published since.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 44 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

26 of 44 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  10. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  11. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  12. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
14 more
  1. 2016Neural Architectures for Named Entity Recognitioncited by 550
  2. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  3. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  4. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  5. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
  6. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 216
  7. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 185
  8. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 144
  9. 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 143
  10. 2018Chinese NER Using Lattice LSTMcited by 81
  11. 2024BEAVER: An Enterprise Benchmark for Text-to-SQLcited by 42
  12. 2025Reliable Text-to-SQL with Adaptive Abstentioncited by 20
  13. 2024TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoringcited by 17
  14. 2025Confidence Estimation for Error Detection in Text-to-SQL Systemscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Correctness: Resolving Underspecification in Agentic Text-to-SQL
  2. 2026Trace Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems
  3. 2026Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems
  4. 2024Text2SQL is Not Enough: Unifying AI and Databases with TAG
  5. 2026"I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generation
  6. 2025Synthetic Clarification and Correction Dialogues about Data-Centric Tasks -- A Teacher-Student Approach
  7. 2026QUIETT: Query-Independent Table Transformation for Robust Reasoning
  8. 2026Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings