mlsanityData to 7 Oct 2026

Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data

arXiv 2106.05006 ↗First posted 9 Jun 2021cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

Most available semantic parsing datasets, comprising of pairs of natural utterances and logical forms, were collected solely for the purpose of training and evaluation of natural language understanding systems. As a result, they do not contain any of the richness and variety of natural-occurring utterances, where humans ask about data they need or are curious about. In this work, we release SEDE, a dataset with 12,023 pairs of utterances and SQL queries collected from real usage on the Stack Exchange website. We show that these pairs contain a variety of real-world challenges which were rarely reflected so far in any other semantic parsing dataset, propose an evaluation metric based on comparison of partial query clauses that is more suitable for real-world queries, and conduct experiments with strong baselines, showing a large gap between the performance on SEDE compared to other common datasets.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 9 Jun 2021.

Builds on

18 of 28 references.

References · 18

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  4. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  5. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  6. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  7. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  8. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  9. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  10. 2018Improving Text-to-SQL Evaluation Methodologycited by 133
  11. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  12. 2020Semantic Evaluation for Text-to-SQL with Distilled Test Suitescited by 123
6 more
  1. 2017Learning a Neural Semantic Parser from User Feedbackcited by 98
  2. 2020Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Trainingcited by 65
  3. 2020Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedbackcited by 44
  4. 2019Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Studycited by 40
  5. 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
  6. 2014Type-Driven Incremental Semantic Parsing with Polymorphismcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 12; 2024: 10; 2025: 12; 2026: 3.

43 papers.

Sort:

By area:

  1. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  2. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 223
  3. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  4. 2024Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Datacited by 61
  5. 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
  6. 2022Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expectcited by 47
  7. 2025DABstep: Data Agent Benchmark for Multi-step Reasoningcited by 44
  8. 2023EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Recordscited by 42
  9. 2023ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systemscited by 42
  10. 2022Prompt Injection: Parameterization of Fixed Inputscited by 35
  11. 2023Benchmarking and Improving Text-to-SQL Generation under Ambiguitycited by 31
  12. 2023Natural Language Interfaces for Tabular Data Querying and Visualization: A Surveycited by 27
31 more
  1. 2023CRUSH4SQL: Collective Retrieval Using Schema Hallucination For Text2SQLcited by 23
  2. 2025BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactionscited by 17
  3. 2024TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoringcited by 15
  4. 2025Towards LLM Agents for Earth Observationcited by 13
  5. 2024Evaluating the Data Model Robustness of Text-to-SQL Systems Based on Real User Queriescited by 12
  6. 2022Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Surveycited by 11
  7. 2023Testing the Limits of Unified Sequence to Sequence LLM Pretraining on Diverse Table Data Taskscited by 10
  8. 2024Vi(E)va LLM! A Conceptual Stack for Evaluating and Interpreting Generative AI-based Visualizationscited by 10
  9. 2024BookSQL: A Large Scale Text-to-SQL Dataset for Accounting Domaincited by 10
  10. 2025Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challengescited by 10
  11. 2025Sparks of Tabular Reasoning via Text2SQL Reinforcement Learningcited by 9
  12. 2022Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parserscited by 7
  13. 2023FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Languagecited by 7
  14. 2024EHR-SeqSQL : A Sequential Text-to-SQL Dataset For Interactively Exploring Electronic Health Recordscited by 5
  15. 2025STRuCT-LLM: Unifying Tabular and Graph Reasoning with Reinforcement Learning for Semantic Parsingcited by 5
  16. 2024Improving Generalization in Semantic Parsing by Increasing Natural Language Variationcited by 4
  17. 2025RubikSQL: Lifelong Learning Agentic Knowledge Base as an Industrial NL2SQL Systemcited by 4
  18. 2022A Survey on Artificial Intelligence for Source Code: A Dialogue Systems Perspectivecited by 3
  19. 2023Structured Case-based Reasoning for Inference-time Adaptation of Text-to-SQL parserscited by 3
  20. 2024Towards Unbiased Evaluation of Detecting Unanswerable Questions in EHRSQLcited by 3
  21. 2025Conversational Lexicography: Querying Lexicographic Data on Knowledge Graphs with SPARQL through Natural Languagecited by 3
  22. 2021Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questionscited by 2
  23. 2023Semantic Parsing for Complex Data Retrieval: Targeting Query Plans vs. SQL for No-Code Access to Relational Databasescited by 1
  24. 2025REFORMER: A ChatGPT-Driven Data Synthesis Framework Elevating Text-to-SQL Modelscited by 1
  25. 2025FinStat2SQL: A Text2SQL Pipeline for Financial Statement Analysiscited by 1
  26. 2025SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gapscited by 1
  27. 2026Schema-First Retrieval: Embedding Catalogs for Natural Language Analyticscited by 1
  28. 2023Domain Adaptation of a State of the Art Text-to-SQL Model: Lessons Learned and Challenges Foundcited by 0
  29. 2025Multi-Dimensional Summarization Agents with Context-Aware Reasoning over Enterprise Tablescited by 0
  30. 2026Natural Language Interfaces for Spatial and Temporal Databases: A Comprehensive Overview of Methods, Taxonomy, and Future Directionscited by 0
  31. 2026Debiasing Without Protected Attributes: Latent Concept Erasure from Textual Profilescited by 0