# A Comparative Survey of Recent Natural Language Interfaces for Databases

arXiv: https://arxiv.org/abs/1906.08990 · HTML: https://mlsanity.com/paper/1906.08990/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2019.

## Built on by (most cited first, 30 of 33)

- [ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems](https://mlsanity.com/paper/2306.04743/) (2023)
- [Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation](https://mlsanity.com/paper/2212.09994/) (2022)
- [Natural Language Interfaces for Tabular Data Querying and Visualization: A Survey](https://mlsanity.com/paper/2310.17894/) (2023)
- [L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models](https://mlsanity.com/paper/2309.17446/) (2023)
- [mRAT-SQL+GAP:A Portuguese Text-to-SQL Transformer](https://mlsanity.com/paper/2110.03546/) (2021)
- [Weakly Supervised Text-to-SQL Parsing through Question Decomposition](https://mlsanity.com/paper/2112.06311/) (2021)
- [Knowledge Graph Question Answering Datasets and Their Generalizability: Are They Enough for Future Research?](https://mlsanity.com/paper/2205.06573/) (2022)
- [Evaluating the Data Model Robustness of Text-to-SQL Systems Based on Real User Queries](https://mlsanity.com/paper/2402.08349/) (2024)
- [DBCopilot: Natural Language Querying over Massive Databases via Schema Routing](https://mlsanity.com/paper/2312.03463/) (2023)
- [Marrying Dialogue Systems with Data Visualization: Interactive Data Visualization Generation from Natural Language Conversations](https://mlsanity.com/paper/2307.16013/) (2023)
- [Speech-to-SQL: Towards Speech-driven SQL Query Generation From Natural Language Question](https://mlsanity.com/paper/2201.01209/) (2022)
- [Knowledge Graph - Deep Learning: A Case Study in Question Answering in Aviation Safety Domain](https://mlsanity.com/paper/2205.15952/) (2022)
- [Knowledge Graph Question Answering for Materials Science (KGQA4MAT): Developing Natural Language Interface for Metal-Organic Frameworks Knowledge Graph (MOF-KG) Using LLM](https://mlsanity.com/paper/2309.11361/) (2023)
- [DBTagger: Multi-Task Learning for Keyword Mapping in NLIDBs Using Bi-Directional Recurrent Neural Networks](https://mlsanity.com/paper/2101.04226/) (2021)
- [There is No Big Brother or Small Brother: Knowledge Infusion in Language Models for Link Prediction and Question Answering](https://mlsanity.com/paper/2301.04013/) (2023)
- [A Multilingual Translator to SQL with Database Schema Pruning to Improve Self-Attention](https://mlsanity.com/paper/2306.14256/) (2023)
- [Spider4SPARQL: A Complex Benchmark for Evaluating Knowledge Graph Question Answering Systems](https://mlsanity.com/paper/2309.16248/) (2023)
- [Translating synthetic natural language to database queries: a polyglot deep learning framework](https://mlsanity.com/paper/2104.07010/) (2021)
- [Charting the Future: Using Chart Question-Answering for Scalable Evaluation of LLM-Driven Data Visualizations](https://mlsanity.com/paper/2409.18764/) (2024)
- [Jackal: A Real-World Execution-Based Benchmark Evaluating Large Language Models on Text-to-JQL Tasks](https://mlsanity.com/paper/2509.23579/) (2025)
- [INODE: Building an End-to-End Data Exploration System in Practice [Extended Vision]](https://mlsanity.com/paper/2104.04194/) (2021)
- [xDBTagger: Explainable Natural Language Interface to Databases Using Keyword Mappings and Schema Graph](https://mlsanity.com/paper/2210.03768/) (2022)
- [KITLM: Domain-Specific Knowledge InTegration into Language Models for Question Answering](https://mlsanity.com/paper/2308.03638/) (2023)
- [Using LLMs for Analyzing AIS Data](https://mlsanity.com/paper/2504.07557/) (2025)
- [Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering](https://mlsanity.com/paper/2603.20004/) (2026)
- [EXPO-SQL: Execution-based Clause-level Policy Optimization for Text-to-SQL](https://mlsanity.com/paper/2606.23693/) (2026)
- [GNN2R: Weakly-Supervised Rationale-Providing Question Answering over Knowledge Graphs](https://mlsanity.com/paper/2312.02317/) (2023)
- [Translating Natural Language Queries to SQL Using the T5 Model](https://mlsanity.com/paper/2312.12414/) (2023)
- [Natural Language Interfaces for Spatial and Temporal Databases: A Comprehensive Overview of Methods, Taxonomy, and Future Directions](https://mlsanity.com/paper/2603.23375/) (2026)
- [Beyond Adoption Intention How Trust in Augmented Analytics Relates to Perceived Decision Quality Among Non-Technical BI Users](https://mlsanity.com/paper/2605.20198/) (2026)

## Abstract (arXiv)

Over the last few years natural language interfaces (NLI) for databases have gained significant traction both in academia and industry. These systems use very different approaches as described in recent survey papers. However, these systems have not been systematically compared against a set of benchmark questions in order to rigorously evaluate their functionalities and expressive power. In this paper, we give an overview over 24 recently developed NLIs for databases. Each of the systems is evaluated using a curated list of ten sample questions to show their strengths and weaknesses. We categorize the NLIs into four groups based on the methodology they are using: keyword-, pattern-, parsing-, and grammar-based NLI. Overall, we learned that keyword-based systems are enough to answer simple questions. To solve more complex questions involving subqueries, the system needs to apply some sort of parsing to identify structural dependencies. Grammar-based systems are overall the most powerful ones, but are highly dependent on their manually designed rules. In addition to providing a systematic analysis of the major systems, we derive lessons learned that are vital for designing NLIs that can answer a wide range of user questions.