# Using LLM to select the right SQL Query from candidates

arXiv: https://arxiv.org/abs/2401.02115 · HTML: https://mlsanity.com/paper/2401.02115/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 22 references that are in the map.
- First posted on arXiv 4 Jan 2024.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Semantic Evaluation for Text-to-SQL with Distilled Test Suites](https://mlsanity.com/paper/2010.02840/) (2020)
- [A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability](https://mlsanity.com/paper/2303.13547/) (2023)
- [SmBoP: Semi-autoregressive Bottom-up Semantic Parsing](https://mlsanity.com/paper/2010.12412/) (2020)
- [RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL](https://mlsanity.com/paper/2205.06983/) (2022)
- [Global Reasoning over Database Structures for Text-to-SQL Parsing](https://mlsanity.com/paper/1908.11214/) (2019)
- [Understanding Image Retrieval Re-Ranking: A Graph Neural Network Perspective](https://mlsanity.com/paper/2012.07620/) (2020)
- [N-Best Hypotheses Reranking for Text-To-SQL Systems](https://mlsanity.com/paper/2210.10668/) (2022)
- [Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing](https://mlsanity.com/paper/2101.01686/) (2021)
- [Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker](https://mlsanity.com/paper/2002.00557/) (2020)
- [Explaining Wrong Queries Using Small Examples](https://mlsanity.com/paper/1904.04467/) (2019)
- [Re-Ranking Words to Improve Interpretability of Automatically Generated Topics](https://mlsanity.com/paper/1903.12542/) (2019)

## Built on by (most cited first, 12 of 12)

- [A Survey of Table Reasoning with Large Language Models](https://mlsanity.com/paper/2402.08259/) (2024)
- [Soft Self-Consistency Improves Language Model Agents](https://mlsanity.com/paper/2402.13212/) (2024)
- [SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQL](https://mlsanity.com/paper/2506.00391/) (2025)
- [Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL](https://mlsanity.com/paper/2402.10663/) (2024)
- [MURRE: Multi-Hop Table Retrieval with Removal for Open-Domain Text-to-SQL](https://mlsanity.com/paper/2402.10666/) (2024)
- [DAC: Decomposed Automation Correction for Text-to-SQL](https://mlsanity.com/paper/2408.08779/) (2024)
- [PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agents](https://mlsanity.com/paper/2604.17653/) (2026)
- [Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities](https://mlsanity.com/paper/2505.23838/) (2025)
- [OmniVLN: Omnidirectional 3D Perception and Token-Efficient LLM Reasoning for Visual-Language Navigation across Air and Ground Platforms](https://mlsanity.com/paper/2603.17351/) (2026)
- [Towards Single-System Illusion in Software-Defined Vehicles -- Automated, AI-Powered Workflow](https://mlsanity.com/paper/2403.14460/) (2024)
- [ScaleDoc: Scaling LLM-based Predicates over Large Document Collections](https://mlsanity.com/paper/2509.12610/) (2025)
- [When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications](https://mlsanity.com/paper/2605.10176/) (2026)

## Abstract (arXiv)

Text-to-SQL models can generate a list of candidate SQL queries, and the best query is often in the candidate list, but not at the top of the list. An effective re-rank method can select the right SQL query from the candidate list and improve the model's performance. Previous studies on code generation automatically generate test cases and use them to re-rank candidate codes. However, automatic test case generation for text-to-SQL is an understudied field. We propose an automatic test case generation method that first generates a database and then uses LLMs to predict the ground truth, which is the expected execution results of the ground truth SQL query on this database. To reduce the difficulty for LLMs to predict, we conduct experiments to search for ways to generate easy databases for LLMs and design easy-to-understand prompts. Based on our test case generation method, we propose a re-rank method to select the right SQL query from the candidate list. Given a candidate list, our method can generate test cases and re-rank the candidate list according to their pass numbers on these test cases and their generation probabilities. The experiment results on the validation dataset of Spider show that the performance of some state-of-the-art models can get a 3.6% improvement after applying our re-rank method.