# The Stretto Execution Engine for LLM-Augmented Data Systems

arXiv: https://arxiv.org/abs/2602.04430 · HTML: https://mlsanity.com/paper/2602.04430/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2026.

## Built on by (most cited first, 3 of 3)

- [Evergreen: Efficient Claim Verification for Semantic Aggregates](https://mlsanity.com/paper/2604.26180/) (2026)
- [Opti-Q: A Constraint-Based Optimization Framework for Multi-LLM Question Planning](https://mlsanity.com/paper/2607.22621/) (2026)
- [Semantic Views: Certified Reuse of LLM Labels Across Semantic Queries](https://mlsanity.com/paper/2610.12768/) (2026)

## Abstract (arXiv)

LLM-augmented data systems enable semantic querying over structured and unstructured data, but executing queries with LLM-powered operators introduces a fundamental runtime-accuracy trade-off. In this paper, we present Stretto, a new execution engine that provides end-to-end query guarantees while efficiently navigating this trade-off in a holistic manner. For this, Stretto formulates query planning as a constrained optimization problem and uses a gradient-based optimizer to jointly select operator implementations and allocate error budgets across pipelines. Moreover, to enable fine-grained execution choices, Stretto introduces a novel idea on how KV-caching can be used to realize a spectrum of different physical operators that transform a sparse design space into a dense continuum of runtime-accuracy trade-offs. Experiments show that Stretto outperforms state-of-the-art systems while consistently meeting quality guarantees.