mlsanityData to 7 Oct 2026

SQLformer: Deep Auto-Regressive Query Graph Generation for Text-to-SQL Translation

arXiv 2310.18376 ↗First posted 27 Oct 2023cs.CL cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

In recent years, the task of text-to-SQL translation, which converts natural language questions into executable SQL queries, has gained significant attention for its potential to democratize data access. Despite its promise, challenges such as adapting to unseen databases and aligning natural language with SQL syntax have hindered widespread adoption. To overcome these issues, we introduce SQLformer, a novel Transformer architecture specifically crafted to perform text-to-SQL translation tasks. Our model predicts SQL queries as abstract syntax trees (ASTs) in an autoregressive way, incorporating structural inductive bias in the encoder and decoder layers. This bias, guided by database table and column selection, aids the decoder in generating SQL query ASTs represented as graphs in a Breadth-First Search canonical order. Our experiments demonstrate that SQLformer achieves state-of-the-art performance across six prominent text-to-SQL benchmarks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

46 of 57 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2017Graph Attention Networkscited by 8,854
  8. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  9. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  10. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  11. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  12. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
34 more
  1. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  2. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  3. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  4. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  5. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  6. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  7. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  8. 2016Language to Logical Form with Neural Attentioncited by 158
  9. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  10. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
  11. 2017SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learningcited by 135
  12. 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
  13. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  14. 2020Relational Graph Attention Network for Aspect-based Sentiment Analysiscited by 104
  15. 2020SmBoP: Semi-autoregressive Bottom-up Semantic Parsingcited by 97
  16. 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
  17. 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
  18. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  19. 2022RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQLcited by 85
  20. 2018TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generationcited by 81
  21. 2019Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questionscited by 80
  22. 2020Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Trainingcited by 65
  23. 2020RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databasescited by 63
  24. 2019Global Reasoning over Database Structures for Text-to-SQL Parsingcited by 43
  25. 2021SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQLcited by 43
  26. 2020Optimizing Deeper Transformers on Small Datasetscited by 41
  27. 2022Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsingcited by 30
  28. 2020IGSQL: Database Schema Interaction Graph Based Neural Model for Context-Dependent Text-to-SQL Generationcited by 29
  29. 2022HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsingcited by 26
  30. 2017An Encoder-Decoder Framework Translating Natural Language to Database Queriescited by 23
  31. 2019X-SQL: reinforce schema representation with contextcited by 22
  32. 2022STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsingcited by 19
  33. 2021Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsingcited by 15
  34. 2021Translating synthetic natural language to database queries: a polyglot deep learning frameworkcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
  2. 2025SchemaGraphSQL: Efficient Schema Linking with Pathfinding Graph Algorithms for Text-to-SQL on Large-Scale Databasescited by 3
  3. 2025Bridging Global Intent with Local Details: A Hierarchical Representation Approach for Semantic Validation in Text-to-SQLcited by 0