mlsanityData to 7 Oct 2026

DBCopilot: Natural Language Querying over Massive Databases via Schema Routing

arXiv 2312.03463 ↗First posted 6 Dec 2023cs.CL cs.DB cs.IRcited by 9

Not on the map. Why?

Abstract (arXiv)

The development of Natural Language Interfaces to Databases (NLIDBs) has been greatly advanced by the advent of large language models (LLMs), which provide an intuitive way to translate natural language (NL) questions into Structured Query Language (SQL) queries. While significant progress has been made in LLM-based NL2SQL, existing approaches face several challenges in real-world scenarios of natural language querying over massive databases. In this paper, we present DBCopilot, a framework that addresses these challenges by employing a compact and flexible copilot model for routing over massive databases. Specifically, DBCopilot decouples schema-agnostic NL2SQL into schema routing and SQL generation. This framework utilizes a single lightweight differentiable search index to construct semantic mappings for massive database schemata, and navigates natural language questions to their target databases and tables in a relation-aware joint retrieval manner. The routed schemata and questions are then fed into LLMs for effective SQL generation. Furthermore, DBCopilot introduces a reverse schema-to-question generation paradigm that can automatically learn and adapt the router over massive databases without manual intervention. Experimental results verify that DBCopilot is a scalable and effective solution for schema-agnostic NL2SQL, providing a significant advance in handling natural language querying over massive databases for NLIDBs.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 30 of its 82 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

30 of 82 references.

References · 30

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  6. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  7. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  8. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  9. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  10. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  11. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  12. 2022Transformer Memory as a Differentiable Search Indexcited by 242
18 more
  1. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  2. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  3. 2020Open Question Answering over Tables and Textcited by 163
  4. 2017SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learningcited by 135
  5. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  6. 2022Evaluating the Text-to-SQL Capabilities of Large Language Modelscited by 101
  7. 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
  8. 2023Learning to Tokenize for Generative Retrievalcited by 66
  9. 2018A Survey on Semantic Parsingcited by 62
  10. 2021GitTables: A Large-Scale Corpus of Relational Tablescited by 44
  11. 2022Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learningcited by 38
  12. 2019A Comparative Survey of Recent Natural Language Interfaces for Databasescited by 33
  13. 2022SANTOS: Relationship-based Semantic Table Union Searchcited by 32
  14. 2023Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generationcited by 24
  15. 2020Efficient Joinable Table Discovery in Data Lakes: A High-Dimensional Similarity-Based Approachcited by 23
  16. 2023Selective Demonstrations for Cross-domain Text-to-SQLcited by 21
  17. 2020Context Dependent Semantic Parsing: A Surveycited by 10
  18. 2023Understanding Differential Search Index for Text Retrievalcited by 9

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 8.

9 papers.

  1. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 46
  2. 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 17
  3. 2025Can LLMs Narrate Tabular Data? An Evaluation Framework for Natural Language Representations of Text-to-SQL System Outputscited by 7
  4. 2025RASL: Retrieval Augmented Schema Linking for Massive Database Text-to-SQLcited by 4
  5. 2025SchemaGraphSQL: Efficient Schema Linking with Pathfinding Graph Algorithms for Text-to-SQL on Large-Scale Databasescited by 3
  6. 2025Same Content, Different Representations: A Controlled Study for Table QAcited by 3
  7. 2025Exploring Multi-Table Retrieval Through Iterative Searchcited by 2
  8. 2024Cooperative SQL Generation for Segmented Databases By Using Multi-functional LLM Agentscited by 1
  9. 2025Bridging Global Intent with Local Details: A Hierarchical Representation Approach for Semantic Validation in Text-to-SQLcited by 0