# Semantic Operators: A Declarative Model for Rich, AI-based Data Processing

arXiv: https://arxiv.org/abs/2407.11418 · HTML: https://mlsanity.com/paper/2407.11418/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 63 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 30 of 30)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Calibration of Pre-trained Transformers](https://mlsanity.com/paper/2003.07892/) (2020)
- [Recursively Summarizing Books with Human Feedback](https://mlsanity.com/paper/2109.10862/) (2021)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [Improving Passage Retrieval with Zero-Shot Question Generation](https://mlsanity.com/paper/2204.07496/) (2022)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning](https://mlsanity.com/paper/2310.03094/) (2023)
- [BooookScore: A systematic exploration of book-length summarization in the era of LLMs](https://mlsanity.com/paper/2310.00785/) (2023)
- [Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes](https://mlsanity.com/paper/2304.09433/) (2023)
- [Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels](https://mlsanity.com/paper/2310.14122/) (2023)
- [TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation](https://mlsanity.com/paper/1804.09769/) (2018)
- [Simple, Robust and Optimal Ranking from Pairwise Comparisons](https://mlsanity.com/paper/1512.08949/) (2015)
- [BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics](https://mlsanity.com/paper/1805.01046/) (2018)
- [A Declarative System for Optimizing AI Workloads](https://mlsanity.com/paper/2405.14696/) (2024)
- [ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data](https://mlsanity.com/paper/2403.04871/) (2024)
- [Optimizing LLM Queries in Relational Data Analytics Workloads](https://mlsanity.com/paper/2403.05821/) (2024)
- [UQE: A Query Engine for Unstructured Databases](https://mlsanity.com/paper/2407.09522/) (2024)
- [Approximate Selection with Guarantees using Proxies](https://mlsanity.com/paper/2004.00827/) (2020)
- [SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models](https://mlsanity.com/paper/2311.09818/) (2023)
- [Accelerating Approximate Aggregation Queries with Expensive Predicates](https://mlsanity.com/paper/2108.06313/) (2021)
- [BioDEX: Large-Scale Biomedical Adverse Drug Event Extraction for Real-World Pharmacovigilance](https://mlsanity.com/paper/2305.13395/) (2023)

## Built on by (most cited first, 30 of 34)

- [Text2SQL is Not Enough: Unifying AI and Databases with TAG](https://mlsanity.com/paper/2408.14717/) (2024)
- [DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis](https://mlsanity.com/paper/2508.20033/) (2025)
- [KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakes](https://mlsanity.com/paper/2506.06541/) (2025)
- [Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrieval](https://mlsanity.com/paper/2410.13765/) (2024)
- [The Design of an LLM-powered Unstructured Analytics System](https://mlsanity.com/paper/2409.00847/) (2024)
- [Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches](https://mlsanity.com/paper/2501.03151/) (2025)
- [Abacus: A Cost-Based Optimizer for Semantic Operator Systems](https://mlsanity.com/paper/2505.14661/) (2025)
- [SemBench: A Benchmark for Semantic Query Processing Engines](https://mlsanity.com/paper/2511.01716/) (2025)
- [Weaver: Interweaving SQL and LLM for Table Reasoning](https://mlsanity.com/paper/2505.18961/) (2025)
- [Deep Research is the New Analytics System: Towards Building the Runtime for AI-Driven Analytics](https://mlsanity.com/paper/2509.02751/) (2025)
- [Batch Query Processing and Optimization for Agentic Workflows](https://mlsanity.com/paper/2509.02121/) (2025)
- [Larch: Learned Query Optimization for Semantic Predicates](https://mlsanity.com/paper/2606.07923/) (2026)
- [Making Prompts First-Class Citizens for Adaptive LLM Pipelines](https://mlsanity.com/paper/2508.05012/) (2025)
- [Querying Databases with Function Calling](https://mlsanity.com/paper/2502.00032/) (2025)
- [A Case for Computing on Unstructured Data](https://mlsanity.com/paper/2509.14601/) (2025)
- [Beyond Fine-Tuning: Effective Strategies for Mitigating Hallucinations in Large Language Models for Data Analytics](https://mlsanity.com/paper/2410.20024/) (2024)
- [DRAMA: Unifying Data Retrieval and Analysis for Open-Domain Analytic Queries](https://mlsanity.com/paper/2510.27238/) (2025)
- [SatIR: Scalable High-Recall Constraint-Satisfaction-Based Information Retrieval for Clinical Trials Matching](https://mlsanity.com/paper/2604.08849/) (2026)
- [Blue Data Intelligence Layer: Streaming Data and Agents for Multi-source Multi-modal Data-Centric Applications](https://mlsanity.com/paper/2604.15233/) (2026)
- [Large Databases Need Small, Open-Weight Language Models](https://mlsanity.com/paper/2606.31808/) (2026)
- [The Stochastic Shift: A New Evaluation Paradigm for Text-to-SQL with AI Operators](https://mlsanity.com/paper/2609.21133/) (2026)
- [RelationalFactQA: A Benchmark for Evaluating Tabular Fact Retrieval from Large Language Models](https://mlsanity.com/paper/2505.21409/) (2025)
- [SABER: A SQL-Compatible Semantic Document Processing System Based on Extended Relational Algebra](https://mlsanity.com/paper/2509.00277/) (2025)
- [Play by the Type Rules: Inferring Constraints for LLM Functions in Declarative Programs](https://mlsanity.com/paper/2509.20208/) (2025)
- [Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement](https://mlsanity.com/paper/2512.05525/) (2025)
- [KathDB: Explainable Multimodal Database Management System with Human-AI Collaboration](https://mlsanity.com/paper/2512.11067/) (2025)
- [Filtered Approximate Nearest Neighbor Search in Vector Databases: System Design and Performance Analysis](https://mlsanity.com/paper/2602.11443/) (2026)
- [SODIUM: From Open Web Data to Queryable Databases](https://mlsanity.com/paper/2603.18447/) (2026)
- [SemJoin: Semantic Join Optimization](https://mlsanity.com/paper/2606.29532/) (2026)
- [Beyond expert users: agents should help users construct preferences, not just elicit them](https://mlsanity.com/paper/2606.30863/) (2026)

## Abstract (arXiv)

The semantic capabilities of large language models (LLMs) have the potential to enable rich analytics and reasoning over vast knowledge corpora. Unfortunately, existing systems either empirically optimize expensive LLM-powered operations with no performance guarantees, or serve a limited set of row-wise LLM operations, providing limited robustness, expressiveness and usability. We introduce semantic operators, the first formalism for declarative and general-purpose AI-based transformations based on natural language specifications (e.g., filtering, sorting, joining or aggregating records using natural language criteria). Each operator opens a rich space for execution plans, similar to relational operators. Our model specifies the expected behavior of each operator with a high-quality gold algorithm, and we develop an optimization framework that reduces cost, while providing accuracy guarantees with respect to a gold algorithm. Using this approach, we propose several novel optimizations to accelerate semantic filtering, joining, group-by and top-k operations by up to 1,000×. We implement semantic operators in the LOTUS system and demonstrate LOTUS' effectiveness on real, bulk-semantic processing applications, including fact-checking, biomedical multi-label classification, search, and topic analysis. We show that the semantic operator model is expressive, capturing state-of-the-art AI pipelines in a few operator calls, and making it easy to express new pipelines that match or exceed quality of recent LLM-based analytic systems by up to 170%, while offering accuracy guarantees. Overall, LOTUS programs match or exceed the accuracy of state-of-the-art AI pipelines for each task while running up to 3.6× faster than the highest-quality baselines. LOTUS is publicly available at https://github.com/lotus-data/lotus.