mlsanityData to 14 Oct 2026

STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing

arXiv 2210.11888 ↗First posted 21 Oct 2022cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a novel SQL guided pre-training framework STAR for context-dependent text-to-SQL parsing, which leverages contextual information to enrich natural language (NL) utterance and table schema representations for text-to-SQL conversations. Concretely, we propose two novel pre-training objectives which respectively explore the context-dependent interactions of NL utterances and SQL queries within each text-to-SQL conversation: (i) schema state tracking (SST) objective that tracks and explores the schema states of context-dependent SQL queries in the form of schema-states by predicting and updating the value of each schema slot during interaction; (ii) utterance dependency tracking (UDT) objective that employs weighted contrastive learning to pull together two semantically similar NL utterances and push away the representations of semantically dissimilar NL utterances within each conversation. In addition, we construct a high-quality large-scale context-dependent text-to-SQL conversation corpus to pre-train STAR. Extensive experiments show that STAR achieves new state-of-the-art performance on two downstream benchmarks (SParC and CoSQL), significantly outperforming previous pre-training methods and ranking first on the leaderboard. We believe the release of the constructed corpus, codebase and pre-trained STAR checkpoints would push forward the research in this area. For reproducibility, we release our code and data at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/star.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

31 of 41 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  6. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  7. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  8. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  9. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
  10. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  11. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  12. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 398
19 more
  1. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 235
  2. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 185
  3. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  4. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 144
  5. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  6. 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
  7. 2019Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questionscited by 80
  8. 2020Grounded Adaptation for Zero-shot Executable Semantic Parsingcited by 76
  9. 2020Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Trainingcited by 65
  10. 2022A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directionscited by 52
  11. 2022S2SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parserscited by 41
  12. 2022Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsingcited by 30
  13. 2020IGSQL: Database Schema Interaction Graph Based Neural Model for Context-Dependent Text-to-SQL Generationcited by 29
  14. 2021Awakening Latent Grounding from Pretrained Language Models for Semantic Parsingcited by 28
  15. 2022HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsingcited by 26
  16. 2021Logic-Consistency Text Generation from Semantic Parsescited by 23
  17. 2020Tracking Interaction States for Multi-Turn Text-to-SQL Semantic Parsingcited by 18
  18. 2021Decoupled Dialogue Modeling and Semantic Parsing for Multi-Turn Text-to-SQLcited by 13
  19. 2022SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parserscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 5; 2025: 2; 2026: 1.

19 papers.

Sort:

By area:

  1. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  2. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 178
  3. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 120
  4. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 119
  5. 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
  6. 2023MultiTabQA: Generating Tabular Answers for Multi-Table Question Answeringcited by 32
  7. 2023Effective Distillation of Table-based Reasoning Ability from LLMscited by 23
  8. 2023Prompting GPT-3.5 for Text-to-SQL with De-semanticization and Skeleton Retrievalcited by 16
  9. 2024CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editionscited by 13
  10. 2023Retrieval-augmented GPT-3.5-based Text-to-SQL Framework with Sample-aware Prompting and Dynamic Revision Chaincited by 12
  11. 2025Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Reviewcited by 8
  12. 2025Plugging Schema Graph into Multi-Table QA: A Human-Guided Framework for Reducing LLM Reliancecited by 7
7 more
  1. 2024EHR-SeqSQL : A Sequential Text-to-SQL Dataset For Interactively Exploring Electronic Health Recordscited by 5
  2. 2023SQLformer: Deep Auto-Regressive Query Graph Generation for Text-to-SQL Translationcited by 3
  3. 2024Filling Memory Gaps: Enhancing Continual Semantic Parsing via SQL Syntax Variance-Guided LLMs without Real Data Replaycited by 3
  4. 2023Correcting Semantic Parses with Natural Language through Dynamic Schema Encodingcited by 2
  5. 2023Exploring the Compositional Generalization in Context Dependent Text-to-SQL Parsingcited by 2
  6. 2024PromptMind Team at EHRSQL-2024: Improving Reliability of SQL Generation using Ensemble LLMscited by 2
  7. 2026Track-SQL: Enhancing Generative Language Models with Dual-Extractive Modules for Schema and Context Tracking in Multi-turn Text-to-SQLcited by 0