mlsanityData to 7 Oct 2026

Structure-Grounded Pretraining for Text-to-SQL

arXiv 2010.12773 ↗First posted 24 Oct 2020cs.CL cs.AIcited by 126

Not on the map. Why?

Abstract (arXiv)

Learning to capture text-table alignment is essential for tasks like text-to-SQL. A model needs to correctly recognize natural language references to columns and values and to ground them in the given database schema. In this paper, we present a novel weakly supervised Structure-Grounded pretraining framework (StruG) for text-to-SQL that can effectively learn to capture text-table alignment based on a parallel text-table corpus. We identify a set of novel prediction tasks: column grounding, value grounding and column-value mapping, and leverage them to pretrain a text-table encoder. Additionally, to evaluate different methods under more realistic text-table alignment settings, we create a new evaluation set Spider-Realistic based on Spider dev set with explicit mentions of column names removed, and adopt eight existing text-to-SQL datasets for cross-database evaluation. STRUG brings significant improvement over BERT-LARGE in all settings. Compared with existing pretraining methods such as GRAPPA, STRUG achieves similar performance on Spider, and outperforms all baselines on more realistic sets. The Spider-Realistic dataset is available at https://doi.org/10.5281/zenodo.5205322.

Key facts. Cited by 126 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 24; 2023: 19; 2024: 25; 2025: 27; 2026: 18.

126 papers.

Sort:

By area:

  1. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  2. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 339
  3. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  4. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  5. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  6. 2022Can Foundation Models Wrangle Your Data?cited by 173
  7. 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
  8. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 118
  9. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  10. 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 105
  11. 2022Evaluating the Text-to-SQL Capabilities of Large Language Modelscited by 101
  12. 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
38 more
  1. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 90
  2. 2022RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQLcited by 85
  3. 2023Exploring Chain-of-Thought Style Prompting for Text-to-SQLcited by 69
  4. 2024A Survey on Employing Large Language Models for Text-to-SQL Taskscited by 67
  5. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thoughtcited by 65
  6. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  7. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  8. 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
  9. 2024Synthesizing Text-to-SQL Data from Weak and Strong LLMscited by 53
  10. 2022A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directionscited by 51
  11. 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 50
  12. 2023Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategiescited by 44
  13. 2021Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Datacited by 43
  14. 2021SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQLcited by 43
  15. 2022Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Taskscited by 42
  16. 2023SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)cited by 42
  17. 2022S2SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parserscited by 41
  18. 2022Prompt Injection: Parameterization of Fixed Inputscited by 35
  19. 2024PURPLE: Making a Large Language Model a Better SQL Writercited by 34
  20. 2024SQLBench: A Comprehensive Evaluation for Text-to-SQL Capabilities of Large Language Modelscited by 32
  21. 2024Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigmcited by 31
  22. 2022OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answeringcited by 29
  23. 2022ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examplescited by 29
  24. 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
  25. 2024A Survey of Useful LLM Evaluationcited by 28
  26. 2021Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalizationcited by 27
  27. 2023Natural Language Interfaces for Tabular Data Querying and Visualization: A Surveycited by 27
  28. 2022HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsingcited by 26
  29. 2025Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewardscited by 25
  30. 2023Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generationcited by 24
  31. 2021ReasonBERT: Pre-trained to Reason with Distant Supervisioncited by 22
  32. 2022Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graphcited by 22
  33. 2022Knowledge Base Question Answering: A Semantic Parsing Perspectivecited by 22
  34. 2025DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AIcited by 22
  35. 2023Text-to-SQL Error Correction with Language Models of Codecited by 21
  36. 2021FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretrainingcited by 19
  37. 2022STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsingcited by 19
  38. 2024Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruningcited by 16

50 most cited of 126 All 126 (JSON)