mlsanityData to 7 Oct 2026

GP: Context-free Grammar Pre-training for Text-to-SQL Parsers

arXiv 2101.09901 ↗First posted 25 Jan 2021cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

A new method for Text-to-SQL parsing, Grammar Pre-training (GP), is proposed to decode deep relations between question and database. Firstly, to better utilize the information of databases, a random value is added behind a question word which is recognized as a column, and the new sentence serves as the model input. Secondly, initialization of vectors for decoder part is optimized, with reference to the former encoding so that question information can be concerned. Finally, a new approach called flooding level is adopted to get the non-zero training loss which can generalize better results. By encoding the sentence with GRAPPA and RAT-SQL model, we achieve better performance on spider, a cross-DB Text-to-SQL dataset (72.8 dev, 69.8 test). Experiments show that our method is easier to converge during training and has excellent robustness.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 22 of its 28 references that are in the map. First posted on arXiv 25 Jan 2021.

Builds on

22 of 28 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2018Deep contextualized word representationscited by 2,285
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2015Pointer Networkscited by 882
  10. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  11. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  12. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
10 more
  1. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  2. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  3. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  4. 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
  5. 2018TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generationcited by 81
  6. 2019Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questionscited by 80
  7. 2018SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Taskcited by 74
  8. 2020Do We Need Zero Training Loss After Achieving Zero Training Error?cited by 61
  9. 2019Global Reasoning over Database Structures for Text-to-SQL Parsingcited by 43
  10. 2019Bridging the Semantic Gap with SQL Query Logs in Natural Language Interfaces to Databasescited by 15

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 1; 2023: 1.

5 papers.

  1. 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
  2. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  3. 2021Hierarchical Neural Data Synthesis for Semantic Parsingcited by 11
  4. 2022T5QL: Taming language models for SQL generationcited by 8
  5. 2021Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questionscited by 2