# Hybrid Querying Over Relational Databases and Large Language Models

arXiv: https://arxiv.org/abs/2408.00884 · HTML: https://mlsanity.com/paper/2408.00884/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [Batch Prompting: Efficient Inference with Large Language Model APIs](https://mlsanity.com/paper/2301.08721/) (2023)
- [Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases](https://mlsanity.com/paper/2403.10446/) (2024)
- [Large Language Models as Data Preprocessors](https://mlsanity.com/paper/2308.16361/) (2023)
- [BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra](https://mlsanity.com/paper/2402.17882/) (2024)
- [Querying Large Language Models with SQL](https://mlsanity.com/paper/2304.00472/) (2023)
- [Revisiting Prompt Engineering via Declarative Crowdsourcing](https://mlsanity.com/paper/2308.03854/) (2023)
- [LLM-SQL-Solver: Can LLMs Determine SQL Equivalence?](https://mlsanity.com/paper/2312.10321/) (2023)

## Built on by (most cited first, 3 of 3)

- [Access Paths for Efficient Ordering with Large Language Models](https://mlsanity.com/paper/2509.00303/) (2025)
- [An advanced AI driven database system](https://mlsanity.com/paper/2507.17778/) (2025)
- [Compositional Online Learning for Semantic Data Processing Systems](https://mlsanity.com/paper/2608.27244/) (2026)

## Abstract (arXiv)

Database queries traditionally operate under the closed-world assumption, providing no answers to questions that require information beyond the data stored in the database. Hybrid querying using SQL offers an alternative by integrating relational databases with large language models (LLMs) to answer beyond-database questions. In this paper, we present the first cross-domain benchmark, SWAN, containing 120 beyond-database questions over four real-world databases. To leverage state-of-the-art language models in addressing these complex questions in SWAN, we present two solutions: one based on schema expansion and the other based on user defined functions. We also discuss optimization opportunities and potential future directions. Our evaluation demonstrates that using GPT-4 Turbo with few-shot prompts, one can achieves up to 40.0% in execution accuracy and 48.2% in data factuality. These results highlights both the potential and challenges for hybrid querying. We believe that our work will inspire further research in creating more efficient and accurate data systems that seamlessly integrate relational databases and large language models to address beyond-database questions.