# Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewards

arXiv: https://arxiv.org/abs/2505.04671 · HTML: https://mlsanity.com/paper/2505.04671/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 58 references that are in the map.
- First posted on arXiv 7 May 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 26 of 26)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [Improve Mathematical Reasoning in Language Models by Automated Process Supervision](https://mlsanity.com/paper/2406.06592/) (2024)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows](https://mlsanity.com/paper/2411.07763/) (2024)
- [C3: Zero-shot Text-to-SQL with ChatGPT](https://mlsanity.com/paper/2307.07306/) (2023)
- [CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL](https://mlsanity.com/paper/2410.01943/) (2024)
- [Structure-Grounded Pretraining for Text-to-SQL](https://mlsanity.com/paper/2010.12773/) (2020)
- [Towards Robustness of Text-to-SQL Models against Synonym Substitution](https://mlsanity.com/paper/2106.01065/) (2021)
- [Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization](https://mlsanity.com/paper/2109.05157/) (2021)
- [OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale](https://mlsanity.com/paper/2503.02240/) (2025)
- [Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselines](https://mlsanity.com/paper/1803.07246/) (2018)
- [SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning](https://mlsanity.com/paper/2504.08600/) (2025)
- [Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL](https://mlsanity.com/paper/2503.23157/) (2025)
- [Robust Text-to-SQL Generation with Execution-Guided Decoding](https://mlsanity.com/paper/1807.03100/) (2018)
- [ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems](https://mlsanity.com/paper/2306.04743/) (2023)
- [ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration](https://mlsanity.com/paper/2502.00675/) (2025)
- [NL2SQL-BUGs: A Benchmark for Detecting Semantic Errors in NL2SQL Translation](https://mlsanity.com/paper/2503.11984/) (2025)
- [ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedback](https://mlsanity.com/paper/2503.19988/) (2025)
- [SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQL](https://mlsanity.com/paper/2502.11741/) (2025)
- [STaR-SQL: Self-Taught Reasoner for Text-to-SQL](https://mlsanity.com/paper/2502.13550/) (2025)
- [EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routing](https://mlsanity.com/paper/2503.22402/) (2025)
- [AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Framework](https://mlsanity.com/paper/2412.10422/) (2024)
- [Graph Neural Networks for Databases: A Survey](https://mlsanity.com/paper/2502.12908/) (2025)

## Built on by (most cited first, 27 of 27)

- [A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models](https://mlsanity.com/paper/2510.08049/) (2025)
- [Agentar-Scale-SQL: Advancing Text-to-SQL through Orchestrated Test-Time Scaling](https://mlsanity.com/paper/2509.24403/) (2025)
- [Reasoning and Tool-use Compete in Agentic RL:From Quantifying Interference to Disentangled Tuning](https://mlsanity.com/paper/2602.00994/) (2026)
- [CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning](https://mlsanity.com/paper/2505.13271/) (2025)
- [Enhancing Large Language Model Reasoning with Reward Models: An Analytical Survey](https://mlsanity.com/paper/2510.01925/) (2025)
- [Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Reward](https://mlsanity.com/paper/2505.12380/) (2025)
- [AgenticDataBench: A Comprehensive Benchmark for Data Agents](https://mlsanity.com/paper/2607.01647/) (2026)
- [Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning](https://mlsanity.com/paper/2604.07941/) (2026)
- [ETS: Energy-Guided Test-Time Scaling for Training-Free RL Alignment](https://mlsanity.com/paper/2601.21484/) (2026)
- [Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis](https://mlsanity.com/paper/2604.24198/) (2026)
- [TABQAWORLD: Optimizing Multimodal Reasoning for Multi-Turn Table Question Answering](https://mlsanity.com/paper/2604.03393/) (2026)
- [Enhancing Table Reasoning with Deterministic Table-State Rewards](https://mlsanity.com/paper/2601.22530/) (2026)
- [Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment](https://mlsanity.com/paper/2607.04728/) (2026)
- [HES-SQL: Hybrid Reasoning for Efficient Text-to-SQL with Structural Skeleton Guidance](https://mlsanity.com/paper/2510.08896/) (2025)
- [Every Step Counts: Step-Level Credit Assignment for Tool-Integrated Text-to-SQL](https://mlsanity.com/paper/2605.04719/) (2026)
- [PRO-CUA: Process-Reward Optimization for Computer Use Agents](https://mlsanity.com/paper/2605.29119/) (2026)
- [Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards](https://mlsanity.com/paper/2606.06825/) (2026)
- [SQLConductor: Search-to-Policy Learning for Step-wise Text-to-SQL Orchestration](https://mlsanity.com/paper/2606.23537/) (2026)
- [BAP-SQL: Budget-Aware Observation Planning for Agentic Text-to-SQL](https://mlsanity.com/paper/2608.02876/) (2026)
- [ROSE: An Intent-Centered Evaluation Metric for NL2SQL](https://mlsanity.com/paper/2604.12988/) (2026)
- [Process Rewards with Learned Reliability](https://mlsanity.com/paper/2605.15529/) (2026)
- [CAPER: Clause-Aligned Process Supervision for Text-to-SQL](https://mlsanity.com/paper/2606.03327/) (2026)
- [From Test-Time Scaling to Reusable Memory: Measuring Crystallization in Text-to-SQL](https://mlsanity.com/paper/2608.07213/) (2026)
- [Data Agents: Agentic Data Systems](https://mlsanity.com/paper/2609.24137/) (2026)
- [Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses](https://mlsanity.com/paper/2609.35255/) (2026)
- [ARCS: Towards Precise Text-to-SQL via Structured Disambiguation](https://mlsanity.com/paper/2610.09396/) (2026)
- [MERIT: Multimodal Evidence Reasoning over Image Tables](https://mlsanity.com/paper/2610.17937/) (2026)

## Abstract (arXiv)

Recent advances in large language models (LLMs) trained with reinforcement learning (RL) have improved Text-to-SQL performance. However, RL-based approaches still struggle with complex queries due to two key limitations: insufficient stepwise execution-aware reasoning grounded in database feedback, and the lack of process-level rewards for guiding reasoning optimization. To address these issues, we propose CoCTE, a divide-and-conquer and execution-aware reasoning framework that progressively composes SQL queries through intermediate view validation and structured Common Table Expressions (CTEs), improving both accuracy and interpretability. To realize a CoCTE reasoning process, we develop Reward-SQL, a unified approach with three stages: (1) model initialization, which equips LLMs with structured CoCTE reasoning capabilities; (2) process reward design, which delivers fine-grained, execution-aware supervision; and (3) process-supervised RL and inference, which integrates process rewards into training and guides the inference stage by process rewards. This paper addresses the core challenges in Reward-SQL and makes the following contributions. We introduce a process reward model (PRM) that combines execution-aware trajectory scoring with entropy-based step weighting, providing dense and interpretable supervision across reasoning steps. We integrate PRM into both RL training and inference stages, stabilizing optimization and improving trajectory exploration with process-level signals. Experiments show that Reward-SQL significantly outperforms baselines with comparable model sizes, and exhibits strong cross-domain generalization.