Learning a Neural Semantic Parser from User Feedback
Not on the map. Why?
Abstract (arXiv)
We present an approach to rapidly and easily build natural language interfaces to databases for new domains, whose performance improves over time based on user feedback, and requires minimal intervention. To achieve this, we adapt neural sequence models to map utterances directly to SQL with its full expressivity, bypassing any intermediate meaning representations. These models are immediately deployed online to solicit feedback from real users to flag incorrect queries. Finally, the popularity of SQL facilitates gathering annotations for incorrect predictions using the crowd, which is directly used to improve our models. This complete feedback loop, without intermediate representations or database specific engineering, opens up new ways of building high quality semantic parsers. Experiments suggest that this approach can be deployed quickly for any new target domain, as we show by learning a semantic parser for an online academic database from scratch.
Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Apr 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 30; 2022: 18; 2023: 29; 2024: 11; 2025: 7; 2026: 2.
98 papers.
Sort:
By area:
- 2023Teaching Large Language Models to Self-Debugcited by 963
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
- 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
- 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
- 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
- 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
- 2022Evaluating the Text-to-SQL Capabilities of Large Language Modelscited by 101
- 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
- 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
- 2023Exploring Chain-of-Thought Style Prompting for Text-to-SQLcited by 69
- 2021Natural SQL: Making SQL Easier to Infer from Natural Language Specificationscited by 66
- 2021Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representationscited by 55
38 more
- 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
- 2022A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directionscited by 51
- 2025OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignmentcited by 51
- 2022Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expectcited by 47
- 2021Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Datacited by 43
- 2023ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systemscited by 42
- 2021Learning to Synthesize Data for Semantic Parsingcited by 41
- 2022S2SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parserscited by 41
- 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
- 2022SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Modelscited by 38
- 2024FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysiscited by 35
- 2025LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architecturescited by 35
- 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
- 2018Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedbackcited by 32
- 2021Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsingcited by 32
- 2023MultiTabQA: Generating Tabular Answers for Multi-Table Question Answeringcited by 32
- 2022Importance of Synthesizing High-quality Data for Text-to-SQL Parsingcited by 27
- 2023Natural Language Interfaces for Tabular Data Querying and Visualization: A Surveycited by 27
- 2021Improving Text-to-SQL with Schema Dependency Learningcited by 25
- 2023A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databasescited by 25
- 2021Structural Information Preserving for Graph-to-Text Generationcited by 24
- 2021Logic-Consistency Text Generation from Semantic Parsescited by 23
- 2022Continual Learning for Instruction Following from Realtime Feedbackcited by 21
- 2023Enhancing Network Management Using Code Generated by Large Language Modelscited by 20
- 2021Zero-Shot Cross-lingual Semantic Parsingcited by 19
- 2023Interactive Text-to-SQL Generation via Editable Step-by-Step Explanationscited by 19
- 2023Learning to Simulate Natural Language Feedback for Interactive Semantic Parsingcited by 19
- 2024SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Mergingcited by 16
- 2021Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsingcited by 15
- 2021mRAT-SQL+GAP:A Portuguese Text-to-SQL Transformercited by 15
- 2025ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelinescited by 15
- 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
- 2022Extrinsic Evaluation of Machine Translation Metricscited by 14
- 2021On The Ingredients of an Effective Zero-shot Semantic Parsercited by 13
- 2023ANPL: Towards Natural Programming with Interactive Decompositioncited by 13
- 2024Archer: A Human-Labeled Text-to-SQL Dataset with Arithmetic, Commonsense and Hypothetical Reasoningcited by 13
- 2023Generating Data for Symbolic Language with Large Language Modelscited by 12
- 2026Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboardscited by 12
50 most cited of 98 All 98 (JSON)
- 2026SQuaD-SQL: Efficient Text-to-SQL with Small Language Models via LLM-Guided Knowledge Distillation9 Jul 2026
- 2026Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards13 Jan 2026
- 2025LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures11 Sep 2025
- 2025Text-to-SQL Oriented to the Process Mining Domain: A PT-EN Dataset for Query Translation18 Aug 2025
- 2025FinStat2SQL: A Text2SQL Pipeline for Financial Statement Analysis29 Jun 2025
- 2025Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequencies27 May 2025
- 2025ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines7 Apr 2025
- 2025Sigma: A dataset for text-to-code semantic parsing with statistical analysis5 Apr 2025
- 2025OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment19 Feb 2025
- 2024Infusing Prompts with Syntax and Semantics8 Dec 2024
- 2024SQLucid: Grounding Natural Language Database Queries with Interactive Explanations10 Sep 2024
- 2024SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging22 Aug 2024
38 more
- 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?9 Aug 2024
- 2024FuncEvalGMN: Evaluating Functional Correctness of SQL via Graph Matching Network9 Jul 2024
- 2024Beyond Text-to-SQL for IoT Defense: A Comprehensive Framework for Querying and Classifying IoT Threats25 Jun 2024
- 2024Improving the Capabilities of Large Language Model Based Marketing Analytics Copilots With Semantic Search And Fine-Tuning16 Apr 2024
- 2024CodeS: Towards Building Open-source Language Models for Text-to-SQL26 Feb 2024
- 2024NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries20 Feb 2024
- 2024Archer: A Human-Labeled Text-to-SQL Dataset with Arithmetic, Commonsense and Hypothetical Reasoning19 Feb 2024
- 2024FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysis19 Jan 2024
- 2023Decoupling SQL Query Hardness Parsing for Text-to-SQL11 Dec 2023
- 2023What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception16 Nov 2023
- 2023A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases13 Nov 2023
- 2023Natural Language Interfaces for Tabular Data Querying and Visualization: A Survey27 Oct 2023
- 2023Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT -- A Text-to-SQL Parsing Comparison16 Oct 2023
- 2023Hermes: Unlocking Security Analysis of Cellular Network Protocols by Synthesizing Finite State Machines from Natural Language Specifications6 Oct 2023
- 2023Text-to-OverpassQL: A Natural Language Interface for Complex Geodata Querying of OpenStreetMap30 Aug 2023
- 2023Enhancing Network Management Using Code Generated by Large Language Models11 Aug 2023
- 2023Adapt and Decompose: Efficient Generalization of Text-to-SQL via Domain Adapted Least-To-Most Prompting1 Aug 2023
- 2023ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems7 Jun 2023
- 2023XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations7 Jun 2023
- 2023ANPL: Towards Natural Programming with Interactive Decomposition29 May 2023
- 2023Federated Learning for Semantic Parsing: Task Formulation, Evaluation Setup, New Algorithms26 May 2023
- 2023CSS: A Large-scale Cross-schema Chinese Text-to-SQL Medical Dataset25 May 2023
- 2023Exploring Chain-of-Thought Style Prompting for Text-to-SQL23 May 2023
- 2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Future23 May 2023
- 2023Generating Data for Symbolic Language with Large Language Models23 May 2023
- 2023MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering22 May 2023
- 2023Continually Improving Extractive QA via Human Feedback21 May 2023
- 2023Learning to Simulate Natural Language Feedback for Interactive Semantic Parsing14 May 2023
- 2023Interactive Text-to-SQL Generation via Editable Step-by-Step Explanations12 May 2023
- 2023Laziness Is a Virtue When It Comes to Compositionality in Neural Semantic Parsing7 May 2023
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs4 May 2023
- 2023Teaching Large Language Models to Self-Debug11 Apr 2023
- 2023Towards Understanding the Generalization of Medical Text-to-SQL Models and Datasets22 Mar 2023
- 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL12 Feb 2023
- 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex30 Jan 2023
- 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing18 Jan 2023
- 2023On the Structural Generalization in Text-to-SQL12 Jan 2023
- 2022Extrinsic Evaluation of Machine Translation Metrics20 Dec 2022
50 newest of 98 All 98 (JSON)
- 2023Teaching Large Language Models to Self-Debugcited by 963
- 2023Exploring Chain-of-Thought Style Prompting for Text-to-SQLcited by 69
- 2023Exploring Chain-of-Thought Style Prompting for Text-to-SQL23 May 2023
- 2023Teaching Large Language Models to Self-Debug11 Apr 2023
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
- 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
- 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
- 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
- 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
- 2022Evaluating the Text-to-SQL Capabilities of Large Language Modelscited by 101
- 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
- 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
- 2021Natural SQL: Making SQL Easier to Infer from Natural Language Specificationscited by 66
- 2021Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representationscited by 55
- 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
- 2022A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directionscited by 51
12 most cited of 93 All 93 (JSON)
- 2026SQuaD-SQL: Efficient Text-to-SQL with Small Language Models via LLM-Guided Knowledge Distillation9 Jul 2026
- 2026Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards13 Jan 2026
- 2025LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures11 Sep 2025
- 2025Text-to-SQL Oriented to the Process Mining Domain: A PT-EN Dataset for Query Translation18 Aug 2025
- 2025FinStat2SQL: A Text2SQL Pipeline for Financial Statement Analysis29 Jun 2025
- 2025Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequencies27 May 2025
- 2025Sigma: A dataset for text-to-code semantic parsing with statistical analysis5 Apr 2025
- 2025OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment19 Feb 2025
- 2024Infusing Prompts with Syntax and Semantics8 Dec 2024
- 2024SQLucid: Grounding Natural Language Database Queries with Interactive Explanations10 Sep 2024
- 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?9 Aug 2024
- 2024FuncEvalGMN: Evaluating Functional Correctness of SQL via Graph Matching Network9 Jul 2024
12 newest of 93 All 93 (JSON)