mlsanityData to 7 Oct 2026

Lucy: Think and Reason to Solve Text-to-SQL

arXiv 2407.05153 ↗First posted 6 Jul 2024cs.AIcited by 0

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Large Language Models (LLMs) have made significant progress in assisting users to query databases in natural language. While LLM-based techniques provide state-of-the-art results on many standard benchmarks, their performance significantly drops when applied to large enterprise databases. The reason is that these databases have a large number of tables with complex relationships that are challenging for LLMs to reason about. We analyze challenges that LLMs face in these settings and propose a new solution that combines the power of LLMs in understanding questions with automated reasoning techniques to handle complex database constraints. Based on these ideas, we have developed a new framework that outperforms state-of-the-art techniques in zero-shot text-to-SQL on complex benchmarks

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 32 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

10 of 32 references.

References · 10

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  3. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  4. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  5. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  6. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 118
  7. 2023How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settingscited by 57
  8. 2023Lemur: Integrating Large Language Models in Automated Program Verificationcited by 49
  9. 2023Ranking LLM-Generated Loop Invariants for Program Verificationcited by 33
  10. 2023A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databasescited by 25

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning
  2. 2025SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs
  3. 2024Multi-Turn Interactions for Text-to-SQL with Large Language Models
  4. 2025SLM-SQL: An Exploration of Small Language Models for Text-to-SQL
  5. 2024Evaluating and Enhancing LLMs for Multi-turn Text-to-SQL with Multiple Question Types
  6. 2025RAISE: Reasoning Agent for Interactive SQL Exploration
  7. 2025Weaver: Interweaving SQL and LLM for Table Reasoning
  8. 2024Open-SQL Framework: Enhancing Text-to-SQL on Open-source Large Language Models