# REFORMER: A ChatGPT-Driven Data Synthesis Framework Elevating Text-to-SQL Models

arXiv: https://arxiv.org/abs/2504.04363 · HTML: https://mlsanity.com/paper/2504.04363/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 31 references that are in the map.
- First posted on arXiv 6 Apr 2025.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://mlsanity.com/paper/2109.05093/) (2021)
- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [SmBoP: Semi-autoregressive Bottom-up Semantic Parsing](https://mlsanity.com/paper/2010.12412/) (2020)
- [Grounded Adaptation for Zero-shot Executable Semantic Parsing](https://mlsanity.com/paper/2009.07396/) (2020)
- [Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training](https://mlsanity.com/paper/2012.10309/) (2020)
- [ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems](https://mlsanity.com/paper/2306.04743/) (2023)
- [Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data](https://mlsanity.com/paper/2106.05006/) (2021)
- [Learning to Synthesize Data for Semantic Parsing](https://mlsanity.com/paper/2104.05827/) (2021)
- [ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser](https://mlsanity.com/paper/2104.04689/) (2021)
- [Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing](https://mlsanity.com/paper/2103.02227/) (2021)
- [Question Generation from SQL Queries Improves Neural Semantic Parsing](https://mlsanity.com/paper/1808.06304/) (2018)
- [Importance of Synthesizing High-quality Data for Text-to-SQL Parsing](https://mlsanity.com/paper/2212.08785/) (2022)
- [Logic-Consistency Text Generation from Semantic Parses](https://mlsanity.com/paper/2108.00577/) (2021)
- [ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning](https://mlsanity.com/paper/2105.12995/) (2021)
- [Hierarchical Neural Data Synthesis for Semantic Parsing](https://mlsanity.com/paper/2112.02212/) (2021)
- [Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parsers](https://mlsanity.com/paper/2210.16613/) (2022)

## Built on by (most cited first, 1 of 1)

- [SemanticAgent: A Semantics-Aware Framework for Text-to-SQL Data Synthesis](https://mlsanity.com/paper/2604.21414/) (2026)

## Abstract (arXiv)

The existing Text-to-SQL models suffer from a shortage of training data, inhibiting their ability to fully facilitate the applications of SQL queries in new domains. To address this challenge, various data synthesis techniques have been employed to generate more diverse and higher quality data. In this paper, we propose REFORMER, a framework that leverages ChatGPT's prowess without the need for additional training, to facilitate the synthesis of (question, SQL query) pairs tailored to new domains. Our data augmentation approach is based on a "retrieve-and-edit" method, where we generate new questions by filling masked question using explanation of SQL queries with the help of ChatGPT. Furthermore, we demonstrate that cycle consistency remains a valuable method of validation when applied appropriately. Our experimental results show that REFORMER consistently outperforms previous data augmentation methods. To further investigate the power of ChatGPT and create a general data augmentation method, we also generate the new data by paraphrasing the question in the dataset and by paraphrasing the description of a new SQL query that is generated by ChatGPT as well. Our results affirm that paraphrasing questions generated by ChatGPT help augment the original data.