# Towards Accurate and Efficient Document Analytics with Large Language Models

arXiv: https://arxiv.org/abs/2405.04674 · HTML: https://mlsanity.com/paper/2405.04674/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2024.

## Built on by (most cited first, 11 of 11)

- [A Declarative System for Optimizing AI Workloads](https://mlsanity.com/paper/2405.14696/) (2024)
- [AgenticData: An Agentic Data Analytics System for Heterogeneous Data](https://mlsanity.com/paper/2508.05002/) (2025)
- [Cortex AISQL: A Production SQL Engine for Unstructured Data](https://mlsanity.com/paper/2511.07663/) (2025)
- [NUDGE: Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval](https://mlsanity.com/paper/2409.02343/) (2024)
- [MinerU-Popo: Universal Post-Processing Model for Structured Document Parsing](https://mlsanity.com/paper/2605.24973/) (2026)
- [CTIFoundry: An Agent-Native Corpus Scaffold for Cyber Threat Intelligence](https://mlsanity.com/paper/2608.18613/) (2026)
- [LLM-Powered Proactive Data Systems](https://mlsanity.com/paper/2502.13016/) (2025)
- [Multi-dimensional Data Analysis and Applications Basing on LLM Agents and Knowledge Graph Interactions](https://mlsanity.com/paper/2510.15258/) (2025)
- [Towards Resource-Efficient Multimodal Intelligence: Learned Routing among Specialized Expert Models](https://mlsanity.com/paper/2511.06441/) (2025)
- [EidosDoc: Implicit Structure Encoding for Cost-Effective Semi-Structured Document QA](https://mlsanity.com/paper/2609.27784/) (2026)
- [SAGE: A Unified Algebra and Self-Adaptive Execution for AI Functions in SQL](https://mlsanity.com/paper/2608.20630/) (2026)

## Abstract (arXiv)

Unstructured data formats account for over 80% of the data currently stored, and extracting value from such formats remains a considerable challenge. In particular, current approaches for managing unstructured documents do not support ad-hoc analytical queries on document collections. Moreover, Large Language Models (LLMs) directly applied to the documents themselves, or on portions of documents through a process of Retrieval-Augmented Generation (RAG), fail to provide high accuracy query results, and in the LLM-only case, additionally incur high costs. Since many unstructured documents in a collection often follow similar templates that impart a common semantic structure, we introduce ZenDB, a document analytics system that leverages this semantic structure, coupled with LLMs, to answer ad-hoc SQL queries on document collections. ZenDB efficiently extracts semantic hierarchical structures from such templatized documents, and introduces a novel query engine that leverages these structures for accurate and cost-effective query execution. Users can impose a schema on their documents, and query it, all via SQL. Extensive experiments on three real-world document collections demonstrate ZenDB's benefits, achieving up to 30% cost savings compared to LLM-based baselines, while maintaining or improving accuracy, and surpassing RAG-based baselines by up to 61% in precision and 80% in recall, at a marginally higher cost.