# N-Best Hypotheses Reranking for Text-To-SQL Systems

arXiv: https://arxiv.org/abs/2210.10668 · HTML: https://mlsanity.com/paper/2210.10668/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://mlsanity.com/paper/2109.05093/) (2021)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://mlsanity.com/paper/2201.05966/) (2022)
- [Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation](https://mlsanity.com/paper/1905.08205/) (2019)
- [CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases](https://mlsanity.com/paper/1909.05378/) (2019)
- [The Curious Case of Hallucinations in Neural Machine Translation](https://mlsanity.com/paper/2104.06683/) (2021)
- [Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?](https://mlsanity.com/paper/2010.12725/) (2020)
- [SParC: Cross-Domain Semantic Parsing in Context](https://mlsanity.com/paper/1906.02285/) (2019)
- [Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing](https://mlsanity.com/paper/2012.12627/) (2020)
- [TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation](https://mlsanity.com/paper/1804.09769/) (2018)
- [Language Model Evaluation Beyond Perplexity](https://mlsanity.com/paper/2106.00085/) (2021)
- [Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing](https://mlsanity.com/paper/1905.06241/) (2019)
- [RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases](https://mlsanity.com/paper/2004.03125/) (2020)
- [Coherence boosting: When your pretrained language model is not paying enough attention](https://mlsanity.com/paper/2110.08294/) (2021)
- [ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser](https://mlsanity.com/paper/2104.04689/) (2021)
- [Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker](https://mlsanity.com/paper/2002.00557/) (2020)
- [SeqGenSQL -- A Robust Sequence Generation Model for Structured Query Language](https://mlsanity.com/paper/2011.03836/) (2020)

## Built on by (most cited first, 19 of 19)

- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents](https://mlsanity.com/paper/2401.00812/) (2024)
- [ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought](https://mlsanity.com/paper/2310.17342/) (2023)
- [A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?](https://mlsanity.com/paper/2408.05109/) (2024)
- [Benchmarking and Improving Text-to-SQL Generation under Ambiguity](https://mlsanity.com/paper/2310.13659/) (2023)
- [PURPLE: Making a Large Language Model a Better SQL Writer](https://mlsanity.com/paper/2403.20014/) (2024)
- [Reliable Text-to-SQL with Adaptive Abstention](https://mlsanity.com/paper/2501.10858/) (2025)
- [Using LLM to select the right SQL Query from candidates](https://mlsanity.com/paper/2401.02115/) (2024)
- [Evaluating the Data Model Robustness of Text-to-SQL Systems Based on Real User Queries](https://mlsanity.com/paper/2402.08349/) (2024)
- [Abacus-SQL: A Text-to-SQL System Empowering Cross-Domain and Open-Domain Database Retrieval](https://mlsanity.com/paper/2504.09824/) (2025)
- [End-to-End Text-to-SQL with Dataset Selection: Leveraging LLMs for Adaptive Query Generation](https://mlsanity.com/paper/2508.06387/) (2025)
- [Conversational Text-to-SQL: An Odyssey into State-of-the-Art and Challenges Ahead](https://mlsanity.com/paper/2302.11054/) (2023)
- [Translating Natural Language Queries to SQL Using the T5 Model](https://mlsanity.com/paper/2312.12414/) (2023)
- [StreamLink: Large-Language-Model Driven Distributed Data Engineering System](https://mlsanity.com/paper/2505.21575/) (2025)
- [Text to Query Plans for Question Answering on Large Tables](https://mlsanity.com/paper/2508.18758/) (2025)
- [DynaQuery: A Self-Adapting Framework for Querying Structured and Multimodal Data](https://mlsanity.com/paper/2510.18029/) (2025)
- [The Nuts and Bolts of Natural Language to SQL Translation: A Systematic Analysis of Model Pipeline Optimisation Approaches and their Interactions](https://mlsanity.com/paper/2607.10911/) (2026)

## Abstract (arXiv)

Text-to-SQL task maps natural language utterances to structured queries that can be issued to a database. State-of-the-art (SOTA) systems rely on finetuning large, pre-trained language models in conjunction with constrained decoding applying a SQL parser. On the well established Spider dataset, we begin with Oracle studies: specifically, choosing an Oracle hypothesis from a SOTA model's 10-best list, yields a 7.7% absolute improvement in both exact match (EM) and execution (EX) accuracy, showing significant potential improvements with reranking. Identifying coherence and correctness as reranking approaches, we design a model generating a query plan and propose a heuristic schema linking algorithm. Combining both approaches, with T5-Large, we obtain a consistent 1%  improvement in EM accuracy, and a ~2.5% improvement in EX, establishing a new SOTA for this task. Our comprehensive error studies on DEV data show the underlying difficulty in making progress on this task.