# Optimizing LLM Queries in Relational Data Analytics Workloads

arXiv: https://arxiv.org/abs/2403.05821 · HTML: https://mlsanity.com/paper/2403.05821/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 9 Mar 2024.

## Builds on (most cited first, 10 of 10)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Prompt Cache: Modular Attention Reuse for Low-Latency Inference](https://mlsanity.com/paper/2311.04934/) (2023)
- [Hydragen: High-Throughput LLM Inference with Shared Prefixes](https://mlsanity.com/paper/2402.05099/) (2024)

## Built on by (most cited first, 22 of 22)

- [Barbarians at the Gate: How AI is Upending Systems Research](https://mlsanity.com/paper/2510.06189/) (2025)
- [A Declarative System for Optimizing AI Workloads](https://mlsanity.com/paper/2405.14696/) (2024)
- [Semantic Operators: A Declarative Model for Rich, AI-based Data Processing](https://mlsanity.com/paper/2407.11418/) (2024)
- [Text2SQL is Not Enough: Unifying AI and Databases with TAG](https://mlsanity.com/paper/2408.14717/) (2024)
- [UQE: A Query Engine for Unstructured Databases](https://mlsanity.com/paper/2407.09522/) (2024)
- [Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution](https://mlsanity.com/paper/2406.00059/) (2024)
- [TQA-Bench: Evaluating LLMs for Multi-Table Question Answering](https://mlsanity.com/paper/2411.19504/) (2024)
- [Top Ten Challenges Towards Agentic Neural Graph Databases](https://mlsanity.com/paper/2501.14224/) (2025)
- [Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective](https://mlsanity.com/paper/2603.16104/) (2026)
- [Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2404.17897/) (2024)
- [Batch Query Processing and Optimization for Agentic Workflows](https://mlsanity.com/paper/2509.02121/) (2025)
- [Beyond Relational: Semantic-Aware Multi-Modal Analytics with LLM-Native Query Optimization](https://mlsanity.com/paper/2511.19830/) (2025)
- [MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints](https://mlsanity.com/paper/2504.09345/) (2025)
- [On-Device LLMs for SMEs: Challenges and Opportunities](https://mlsanity.com/paper/2410.16070/) (2024)
- [Let the Barbarians In: How AI Can Accelerate Systems Performance Research](https://mlsanity.com/paper/2512.14806/) (2025)
- [stratum: A System Infrastructure for Massive Agent-Centric ML Workloads](https://mlsanity.com/paper/2603.03589/) (2026)
- [Beyond Linear LLM Invocation: An Efficient and Effective Semantic Filter Paradigm](https://mlsanity.com/paper/2603.04799/) (2026)
- [AIChilles: Automatically Uncovering Hidden Weaknesses in AI-Evolved Systems](https://mlsanity.com/paper/2606.15834/) (2026)
- [Evergreen: Efficient Claim Verification for Semantic Aggregates](https://mlsanity.com/paper/2604.26180/) (2026)
- [PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers](https://mlsanity.com/paper/2605.02189/) (2026)
- [Semantic Views: Certified Reuse of LLM Labels Across Semantic Queries](https://mlsanity.com/paper/2610.12768/) (2026)
- [Same Query, Different Answer: Run-to-Run Variance in Semantic Query Processing](https://mlsanity.com/paper/2610.13823/) (2026)

## Abstract (arXiv)

Batch data analytics is a growing application for Large Language Models (LLMs). LLMs enable users to perform a wide range of natural language tasks, such as classification, entity extraction, and translation, over large datasets. However, LLM inference is highly costly and slow: for example, an NVIDIA L4 GPU running Llama3-8B can only process 6 KB of text per second, taking about a day to handle 15 GB of data; processing a similar amount of data costs around 10K on OpenAI's GPT-4o. In this paper, we propose novel techniques that can significantly reduce the cost of LLM calls for relational data analytics workloads. Our key contribution is developing efficient algorithms for reordering the rows and the fields within each row of an input table to maximize key-value (KV) cache reuse when performing LLM serving. As such, our approach can be easily applied to existing analytics systems and serving platforms. Our evaluation shows that our solution can yield up to 3.4x improvement in job completion time on a benchmark of diverse LLM-based queries using Llama 3 models. Our solution also achieves a 32% cost savings under OpenAI and Anthropic pricing models.