mlsanityData to 7 Oct 2026

ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration

arXiv 2502.00675 ↗First posted 2 Feb 2025cs.CLcited by 34

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

We present ReFoRCE, a Text-to-SQL agent that tops the Spider 2.0 leaderboard--a challenging benchmark reflecting complex, real-world Text-to-SQL scenarios. While Text-to-SQL systems enable natural language queries over structured databases, deploying them in enterprise environments remains difficult due to large, complex schemas (with over 1,000 columns), diverse SQL dialects (e.g., BigQuery, Snowflake), and sophisticated query requirements (e.g., transformations and analytics). ReFoRCE addresses these challenges through: (a) database information compression via pattern-based table grouping and LLM-guided schema linking to alleviate long-context issues; (b) self-refinement to iteratively correct syntax and semantic errors across dialects; (c) majority-vote consensus to select high-confidence candidates while deferring ambiguous cases arising from sophisticated queries; and (d) iterative column exploration guided by execution feedback to resolve those deferred cases. ReFoRCE achieves new state-of-the-art results, with scores of 35.83 on Spider 2.0-Snow and 36.56 on Spider 2.0-Lite.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 38 references that are in the map. First posted on arXiv 2 Feb 2025.

Builds on

30 of 38 references.

References · 30

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2023Teaching Large Language Models to Self-Debugcited by 963
  10. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  11. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  12. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
18 more
  1. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  2. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  3. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  4. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
  5. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  6. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  7. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 223
  8. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  9. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  10. 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 164
  11. 2024CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQLcited by 132
  12. 2024CodeR: Issue Resolving with Multi-Agent and Task Graphscited by 89
  13. 2024The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Modelscited by 67
  14. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thoughtcited by 65
  15. 2024FlowMind: Automatic Workflow Generation with LLMscited by 30
  16. 2023Flows: Building Blocks of Reasoning and Collaborating AIcited by 25
  17. 2023CRUSH4SQL: Collective Retrieval Using Schema Hallucination For Text2SQLcited by 23
  18. 2024Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilitiescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 27.

34 papers.

Sort:

By area:

  1. 2025Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQLcited by 38
  2. 2025Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewardscited by 25
  3. 2025AgenticData: An Agentic Data Analytics System for Heterogeneous Datacited by 22
  4. 2025Reasoning-Table: Exploring Reinforcement Learning for Table Reasoningcited by 18
  5. 2026AgentSM: Semantic Memory for Agentic Text-to-SQLcited by 14
  6. 2026Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboardscited by 12
  7. 2025Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generationcited by 8
  8. 2026DataSTORM: Deep Research on Large-Scale Databases using Exploratory Data Analysis and Data Storytellingcited by 7
  9. 2026LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Votingcited by 2
  10. 2026AgentNLQ: A General-Purpose Agent for Natural Language to SQLcited by 2
  11. 2026A Semantic-Layer-Mediated Agent for Natural Language to SQL over Heterogeneous Enterprise Databasescited by 2
  12. 2025Declarative Techniques for NL Queries over Heterogeneous Datacited by 1
22 more
  1. 2026Both Ends Count! Just How Good are LLM Agents at "Text-to-Big SQL"?cited by 1
  2. 2026Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineeringcited by 1
  3. 2026EviLink: Multi-Path Schema Linking with Uncertainty-Guided Evidence Acquisition for Large-Scale Text-to-SQLcited by 1
  4. 2026Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving LLM Agentscited by 1
  5. 2025AskDB: An LLM Agent for Natural Language Interaction with Relational Databasescited by 0
  6. 2026RingSQL: Schema-Independent Synthetic Data Generation for Text-to-SQL Reinforcement Learningcited by 0
  7. 2026OptiSQL: Executable SQL Generation from Optical Tokenscited by 0
  8. 2026IESR:Efficient MCTS-Based Modular Reasoning for Text-to-SQL with Large Language Modelscited by 0
  9. 2026SQL Query Engine: A Self-Healing LLM Pipeline for Natural Language to PostgreSQL Translationcited by 0
  10. 2026DisasterLex: An Expert Concept-to-Schema Knowledge Graph for Geospatial Reasoning in Disaster Analyticscited by 0
  11. 2026Sophrosyne: Agentic Exploration of Relational Data Systems Needs Moderationcited by 0
  12. 2026ProSPy: A Profiling-Driven SQL-Python Agentic Framework for Enterprise Text-to-SQLcited by 0
  13. 2026SOMA-SQL: Resolving Multi-Source Ambiguity in NL-to-SQL via Synthetic Log and Execution Probingcited by 0
  14. 2026Data Intelligence Agents: Interpreting, Modeling, and Querying Enterprise Data via Autonomous Coding Agentscited by 0
  15. 2026EvoSQL: Memory-Augmented Critic-Generator Co-Evolution for Text-to-SQLcited by 0
  16. 2026MDB-Link: Hierarchical Schema Linking for Multi-Database Text-to-SQLcited by 0
  17. 2026DexterSQL: Deep Schema Exploration and Rule-based Correction for Text-to-SQL Generationcited by 0
  18. 2026Beyond the Harness: End-to-End Optimization of Context Artifacts for Enterprise Text-to-SQLcited by 0
  19. 2026A Cost-Aware Agentic Architecture for NL-to-SQL over Nested Enterprise Schemas, with a New Benchmarkcited by 0
  20. 2026What Drives Recovery in Agentic Text-to-Cypher? LAST-CQ: An LLM Agent Self-Refinement Frameworkcited by 0
  21. 2026Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agentscited by 0
  22. 2026HarnessSQL: Harness-Native Training for SQL Agents in Realistic Database Environmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQL
  2. 2024SQLFixAgent: Towards Semantic-Accurate Text-to-SQL Parsing via Consistency-Enhanced Multi-Agent Collaboration
  3. 2025SQL-of-Thought: Multi-agentic Text-to-SQL with Guided Error Correction
  4. 2025BAPPA: Benchmarking Agents, Plans, and Pipelines for Automated Text-to-SQL Generation
  5. 2026ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models
  6. 2026FlexSQL: Flexible Exploration and Execution Make Better Text-to-SQL Agents
  7. 2026RoboPhD: Self-Improving Text-to-SQL Through Autonomous Agent Evolution
  8. 2026PExA: Parallel Exploration Agent for Complex Text-to-SQL