mlsanityData to 14 Oct 2026

Semantic Integrity Constraints: Declarative Guardrails for AI-Augmented Data Processing Systems

arXiv 2503.00600 ↗First posted 1 Mar 2025cs.DB cs.AI cs.CLcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

AI-augmented data processing systems (DPSs) integrate large language models (LLMs) into query pipelines, allowing powerful semantic operations on structured and unstructured data. However, the reliability (a.k.a. trust) of these systems is fundamentally challenged by the potential for LLMs to produce errors, limiting their adoption in critical domains. To help address this reliability bottleneck, we introduce semantic integrity constraints (SICs) -- a declarative abstraction for specifying and enforcing correctness conditions over LLM outputs in semantic queries. SICs generalize traditional database integrity constraints to semantic settings, supporting common types of constraints, such as grounding, soundness, and exclusion, with both reactive and proactive enforcement strategies. We argue that SICs provide a foundation for building reliable and auditable AI-augmented data systems. Specifically, we present a system design for integrating SICs into query planning and runtime execution and discuss its realization in AI-augmented DPSs. To guide and evaluate our vision, we outline several design goals -- covering criteria around expressiveness, runtime semantics, integration, performance, and enterprise-scale applicability -- and discuss how our framework addresses each, along with open research challenges.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 27 of its 70 references that are in the map. First posted on arXiv 1 Mar 2025.

Builds on

27 of 70 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024On Calibration of LLM-based Guard Models for Reliable Content Moderation14 Oct

Other references · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2018How Powerful are Graph Neural Networks?cited by 4,393
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  10. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  11. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  12. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
14 more
  1. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  2. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 251
  3. 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 144
  4. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  5. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  6. 2024Automata-based constraints for language model decodingcited by 51
  7. 2024"We Need Structured Output": Towards User-centered Constraints on Large Language Model Outputcited by 46
  8. 2022Validating Large Language Models with ReLMcited by 33
  9. 2023DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelinescited by 27
  10. 2020Model Assertions for Monitoring and Improving ML Modelscited by 22
  11. 2024UQE: A Query Engine for Unstructured Databasescited by 21
  12. 2023CAESURA: Language Models as Multi-Modal Query Plannerscited by 20
  13. 2024The Design of an LLM-powered Unstructured Analytics Systemcited by 20
  14. 2024SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelinescited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Trace Integrity for LLM Data Agents: A Vision for Auditable Structured Reasoning in Real-World Systems
  2. 2025QUITE: A Query Rewrite System Beyond Rules with LLM Agents
  3. 2026PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agents
  4. 2026A Semantic-Layer-Mediated Agent for Natural Language to SQL over Heterogeneous Enterprise Databases
  5. 2024A Simple and Fast Way to Handle Semantic Errors in Transactions
  6. 2026Semantic Caching and Intent-Driven Context Optimization for Multi-Agent Natural Language to Code Systems
  7. 2026Never the Number: Structural Abstention for AI Systems Whose Answers Are Consumed as Fact
  8. 2025LLM and Agent-Driven Data Analysis: A Systematic Approach for Enterprise Applications and System-level Deployment