# JudgeSQL: Reasoning over SQL Candidates with Weighted Consensus Tournament

arXiv: https://arxiv.org/abs/2510.15560 · HTML: https://mlsanity.com/paper/2510.15560/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 6 of its 54 references that are in the map.
- First posted on arXiv 17 Oct 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 6 of 6)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [CHESS: Contextual Harnessing for Efficient SQL Synthesis](https://mlsanity.com/paper/2405.16755/) (2024)
- [J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learning](https://mlsanity.com/paper/2505.10320/) (2025)
- [Semantic Parsing with Syntax- and Table-Aware SQL Generation](https://mlsanity.com/paper/1804.08338/) (2018)
- [Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages](https://mlsanity.com/paper/2506.00912/) (2025)

## Built on by (most cited first, 1 of 1)

- [Replacing Training with Memory: Listwise Selection for Text-to-SQL](https://mlsanity.com/paper/2609.00834/) (2026)

## Abstract (arXiv)

Text-to-SQL is a pivotal task that bridges natural language understanding and structured data access, yet it remains fundamentally challenging due to semantic ambiguity and complex compositional reasoning. While large language models (LLMs) have greatly advanced SQL generation though prompting, supervised finetuning and reinforced tuning, the shift toward test-time scaling exposes a new bottleneck: selecting the correct query from a diverse candidate pool. Existing selection approaches, such as self-consistency or best-of-N decoding, provide only shallow signals, making them prone to inconsistent scoring, fragile reasoning chains, and a failure to capture fine-grained semantic distinctions between closely related SQL candidates. To this end, we introduce JudgeSQL, a principled framework that redefines SQL candidate selection through structured reasoning and weighted consensus tournament mechanism. JudgeSQL develops a reasoning-based SQL judge model that distills reasoning traces with reinforcement learning guided by verifiable rewards, enabling accurate and interpretable judgments. Building on this, a weighted consensus tournament integrates explicit reasoning preferences with implicit generator confidence, yielding selections that are both more reliable and more efficient. Extensive experiments on the BIRD benchmark demonstrate that JudgeSQL exhibits superior SQL judgment capabilities and good cross-scale generalization and robustness to generator capacity.