# ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration

arXiv: https://arxiv.org/abs/2502.00675 · HTML: https://mlsanity.com/paper/2502.00675/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 2 Feb 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 30)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024)
- [PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://mlsanity.com/paper/2109.05093/) (2021)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023)
- [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows](https://mlsanity.com/paper/2411.07763/) (2024)
- [RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL](https://mlsanity.com/paper/2302.05965/) (2023)
- [CodeS: Towards Building Open-source Language Models for Text-to-SQL](https://mlsanity.com/paper/2402.16347/) (2024)
- [CHESS: Contextual Harnessing for Efficient SQL Synthesis](https://mlsanity.com/paper/2405.16755/) (2024)
- [CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL](https://mlsanity.com/paper/2410.01943/) (2024)
- [CodeR: Issue Resolving with Multi-Agent and Task Graphs](https://mlsanity.com/paper/2406.01304/) (2024)
- [The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models](https://mlsanity.com/paper/2408.07702/) (2024)
- [ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought](https://mlsanity.com/paper/2310.17342/) (2023)
- [FlowMind: Automatic Workflow Generation with LLMs](https://mlsanity.com/paper/2404.13050/) (2024)
- [Flows: Building Blocks of Reasoning and Collaborating AI](https://mlsanity.com/paper/2308.01285/) (2023)
- [CRUSH4SQL: Collective Retrieval Using Schema Hallucination For Text2SQL](https://mlsanity.com/paper/2311.01173/) (2023)
- [Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilities](https://mlsanity.com/paper/2411.16742/) (2024)

## Built on by (most cited first, 30 of 37)

- [Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL](https://mlsanity.com/paper/2505.20315/) (2025)
- [Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewards](https://mlsanity.com/paper/2505.04671/) (2025)
- [AgenticData: An Agentic Data Analytics System for Heterogeneous Data](https://mlsanity.com/paper/2508.05002/) (2025)
- [Reasoning-Table: Exploring Reinforcement Learning for Table Reasoning](https://mlsanity.com/paper/2506.01710/) (2025)
- [AgentSM: Semantic Memory for Agentic Text-to-SQL](https://mlsanity.com/paper/2601.15709/) (2026)
- [Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards](https://mlsanity.com/paper/2601.08778/) (2026)
- [Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generation](https://mlsanity.com/paper/2511.21402/) (2025)
- [DataSTORM: Deep Research on Large-Scale Databases using Exploratory Data Analysis and Data Storytelling](https://mlsanity.com/paper/2604.06474/) (2026)
- [LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Voting](https://mlsanity.com/paper/2601.17942/) (2026)
- [AgentNLQ: A General-Purpose Agent for Natural Language to SQL](https://mlsanity.com/paper/2605.19010/) (2026)
- [A Semantic-Layer-Mediated Agent for Natural Language to SQL over Heterogeneous Enterprise Databases](https://mlsanity.com/paper/2606.31041/) (2026)
- [Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agents](https://mlsanity.com/paper/2607.12790/) (2026)
- [Declarative Techniques for NL Queries over Heterogeneous Data](https://mlsanity.com/paper/2510.16470/) (2025)
- [Both Ends Count! Just How Good are LLM Agents at "Text-to-Big SQL"?](https://mlsanity.com/paper/2602.21480/) (2026)
- [Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering](https://mlsanity.com/paper/2603.20004/) (2026)
- [EviLink: Multi-Path Schema Linking with Uncertainty-Guided Evidence Acquisition for Large-Scale Text-to-SQL](https://mlsanity.com/paper/2605.29670/) (2026)
- [MDB-Link: Hierarchical Schema Linking for Multi-Database Text-to-SQL](https://mlsanity.com/paper/2608.09588/) (2026)
- [AskDB: An LLM Agent for Natural Language Interaction with Relational Databases](https://mlsanity.com/paper/2511.16131/) (2025)
- [RingSQL: Schema-Independent Synthetic Data Generation for Text-to-SQL Reinforcement Learning](https://mlsanity.com/paper/2601.05451/) (2026)
- [OptiSQL: Executable SQL Generation from Optical Tokens](https://mlsanity.com/paper/2601.13695/) (2026)
- [IESR:Efficient MCTS-Based Modular Reasoning for Text-to-SQL with Large Language Models](https://mlsanity.com/paper/2602.05385/) (2026)
- [SQL Query Engine: A Self-Healing LLM Pipeline for Natural Language to PostgreSQL Translation](https://mlsanity.com/paper/2604.16511/) (2026)
- [DisasterLex: An Expert Concept-to-Schema Knowledge Graph for Geospatial Reasoning in Disaster Analytics](https://mlsanity.com/paper/2605.30538/) (2026)
- [Sophrosyne: Agentic Exploration of Relational Data Systems Needs Moderation](https://mlsanity.com/paper/2605.30862/) (2026)
- [ProSPy: A Profiling-Driven SQL-Python Agentic Framework for Enterprise Text-to-SQL](https://mlsanity.com/paper/2606.05836/) (2026)
- [SOMA-SQL: Resolving Multi-Source Ambiguity in NL-to-SQL via Synthetic Log and Execution Probing](https://mlsanity.com/paper/2606.11424/) (2026)
- [Data Intelligence Agents: Interpreting, Modeling, and Querying Enterprise Data via Autonomous Coding Agents](https://mlsanity.com/paper/2606.19319/) (2026)
- [Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows](https://mlsanity.com/paper/2607.06229/) (2026)
- [EvoSQL: Memory-Augmented Critic-Generator Co-Evolution for Text-to-SQL](https://mlsanity.com/paper/2607.20489/) (2026)
- [DexterSQL: Deep Schema Exploration and Rule-based Correction for Text-to-SQL Generation](https://mlsanity.com/paper/2608.11889/) (2026)

## Abstract (arXiv)

We present ReFoRCE, a Text-to-SQL agent that tops the Spider 2.0 leaderboard--a challenging benchmark reflecting complex, real-world Text-to-SQL scenarios. While Text-to-SQL systems enable natural language queries over structured databases, deploying them in enterprise environments remains difficult due to large, complex schemas (with over 1,000 columns), diverse SQL dialects (e.g., BigQuery, Snowflake), and sophisticated query requirements (e.g., transformations and analytics). ReFoRCE addresses these challenges through: (a) database information compression via pattern-based table grouping and LLM-guided schema linking to alleviate long-context issues; (b) self-refinement to iteratively correct syntax and semantic errors across dialects; (c) majority-vote consensus to select high-confidence candidates while deferring ambiguous cases arising from sophisticated queries; and (d) iterative column exploration guided by execution feedback to resolve those deferred cases. ReFoRCE achieves new state-of-the-art results, with scores of 35.83 on Spider 2.0-Snow and 36.56 on Spider 2.0-Lite.