# SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising

arXiv: https://arxiv.org/abs/2105.07911 · HTML: https://mlsanity.com/paper/2105.07911/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 58 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [The Natural Language Decathlon: Multitask Learning as Question Answering](https://mlsanity.com/paper/1806.08730/) (2018)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Language to Logical Form with Neural Attention](https://mlsanity.com/paper/1601.01280/) (2016)
- [SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning](https://mlsanity.com/paper/1711.04436/) (2017)
- [Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing](https://mlsanity.com/paper/2012.12627/) (2020)
- [Data Recombination for Neural Semantic Parsing](https://mlsanity.com/paper/1606.03622/) (2016)
- [NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System](https://mlsanity.com/paper/1802.08979/) (2018)
- [A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization](https://mlsanity.com/paper/1902.01069/) (2019)
- [TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation](https://mlsanity.com/paper/1804.09769/) (2018)
- [Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2001.11458/) (2020)
- [Robust Text-to-SQL Generation with Execution-Guided Decoding](https://mlsanity.com/paper/1807.03100/) (2018)
- [Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2005.04816/) (2020)
- [Dynamic Coattention Networks For Question Answering](https://mlsanity.com/paper/1611.01604/) (2016)
- [Machine Comprehension Using Match-LSTM and Answer Pointer](https://mlsanity.com/paper/1608.07905/) (2016)
- [Hybrid Ranking Network for Text-to-SQL](https://mlsanity.com/paper/2008.04759/) (2020)
- [Improving Text-to-SQL with Schema Dependency Learning](https://mlsanity.com/paper/2103.04399/) (2021)

## Built on by (most cited first, 27 of 27)

- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation](https://mlsanity.com/paper/2405.07467/) (2024)
- [A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions](https://mlsanity.com/paper/2208.13629/) (2022)
- [Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect](https://mlsanity.com/paper/2208.10099/) (2022)
- [DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models](https://mlsanity.com/paper/2402.01117/) (2024)
- [StructLM: Towards Building Generalist Models for Structured Knowledge Grounding](https://mlsanity.com/paper/2402.16671/) (2024)
- [PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency](https://mlsanity.com/paper/2403.09732/) (2024)
- [UniSAr: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQL](https://mlsanity.com/paper/2203.07781/) (2022)
- [Reliable Text-to-SQL with Adaptive Abstention](https://mlsanity.com/paper/2501.10858/) (2025)
- [UNITE: A Unified Benchmark for Text-to-SQL Evaluation](https://mlsanity.com/paper/2305.16265/) (2023)
- [Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Survey](https://mlsanity.com/paper/2208.04415/) (2022)
- [Learn from Yesterday: A Semi-Supervised Continual Learning Method for Supervision-Limited Text-to-SQL Task Streams](https://mlsanity.com/paper/2211.11226/) (2022)
- [PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL](https://mlsanity.com/paper/2409.14082/) (2024)
- [Table as a Modality for Large Language Models](https://mlsanity.com/paper/2512.00947/) (2025)
- [DataFrame QA: A Universal LLM Framework on DataFrame Question Answering Without Data Exposure](https://mlsanity.com/paper/2401.15463/) (2024)
- [Lyra: A Benchmark for Turducken-Style Code Generation](https://mlsanity.com/paper/2108.12144/) (2021)
- [CodexDB: Generating Code for Processing SQL Queries using GPT-3 Codex](https://mlsanity.com/paper/2204.08941/) (2022)
- [Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsing](https://mlsanity.com/paper/2210.01425/) (2022)
- [Parameterizing Context: Unleashing the Power of Parameter-Efficient Fine-Tuning and In-Context Tuning for Continual Table Semantic Parsing](https://mlsanity.com/paper/2310.04801/) (2023)
- [SelECT-SQL: Self-correcting ensemble Chain-of-Thought for Text-to-SQL](https://mlsanity.com/paper/2409.10007/) (2024)
- [Linking-Enhanced Pre-Training for Table Semantic Parsing](https://mlsanity.com/paper/2111.09486/) (2021)
- [NeuroQL: A Neuro-Symbolic Language and Dataset for Inter-Subjective Reasoning](https://mlsanity.com/paper/2303.07146/) (2023)
- [Schema-Aware Multi-Task Learning for Complex Text-to-SQL](https://mlsanity.com/paper/2403.09706/) (2024)
- [Extending Confidence-Based Text2Cypher with Grammar and Schema Aware Filtering](https://mlsanity.com/paper/2605.10318/) (2026)
- [Precognition in Task-oriented Dialogue Understanding: Posterior Regularization by Future Context](https://mlsanity.com/paper/2203.03244/) (2022)
- [Domain Specific Question to SQL Conversion with Embedded Data Balancing Technique](https://mlsanity.com/paper/2504.08753/) (2025)
- [LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL](https://mlsanity.com/paper/2510.02350/) (2025)

## Abstract (arXiv)

In text-to-SQL task, seq-to-seq models often lead to sub-optimal performance due to limitations in their architecture. In this paper, we present a simple yet effective approach that adapts transformer-based seq-to-seq model to robust text-to-SQL generation. Instead of inducing constraint to decoder or reformat the task as slot-filling, we propose to train seq-to-seq model with Schema aware Denoising (SeaD), which consists of two denoising objectives that train model to either recover input or predict output from two novel erosion and shuffle noises. These denoising objectives acts as the auxiliary tasks for better modeling the structural data in S2S generation. In addition, we improve and propose a clause-sensitive execution guided (EG) decoding strategy to overcome the limitation of EG decoding for generative model. The experiments show that the proposed method improves the performance of seq-to-seq model in both schema linking and grammar correctness and establishes new state-of-the-art on WikiSQL benchmark. The results indicate that the capacity of vanilla seq-to-seq architecture for text-to-SQL may have been under-estimated.