HarnessSQL: Harness-Native Training for SQL Agents in Realistic Database Environments
Not on the map. Why?
Abstract (arXiv)
Text-to-SQL models are commonly trained to map questions directly to static queries, whereas real-world database agents operate through stateful, multi-turn interaction with live databases -- inspecting schemas, executing probe queries, diagnosing errors, and revising hypotheses. This creates a critical train-deploy mismatch, as the execution harness that mediates this interaction is introduced only at inference time. To bridge this gap, we propose HarnessSQL, a harness-native post-training framework that preserves the full interaction structure throughout both supervised fine-tuning and reinforcement learning. HarnessSQL builds isolated, executable database environments paired with hidden execution oracles, rolls out teachers directly inside the target SQL harness, and retains only verified trajectories for full-sequence SFT, followed by execution-reward RL. Across Spider 2.0-SQLite, HarnessSQL dramatically boosts the execution accuracy of compact models, raising Qwen3-8B from 15.5% to 45.2% and Qwen3-14B from 22.2% to 54.8%, while transferring effectively to out-of-distribution interactive benchmarks such as BIRD-Interact and LiveSQLBench. Our findings demonstrate that training database agents directly within their execution harness is essential for mastering complex, long-horizon database workflows.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 8 Oct 2026.
Builds on
20 of 37 references.
References · 20
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2025Group Sequence Policy Optimizationcited by 788
- 2024Executable Code Actions Elicit Better LLM Agentscited by 537
- 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
- 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
- 2023FireAct: Toward Language Agent Fine-tuningcited by 247
- 2024ToolACE: Winning the Points of LLM Function Callingcited by 204
- 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
- 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 164
8 more
- 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 88
- 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflowscited by 44
- 2025ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Explorationcited by 34
- 2025BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactionscited by 17
- 2025SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applicationscited by 15
- 2025MTIR-SQL: Multi-turn Tool-Integrated Reasoning Reinforcement Learning for Text-to-SQLcited by 10
- 2026FlexSQL: Flexible Exploration and Execution Make Better Text-to-SQL Agentscited by 3
- 2026Does Synthetic Layered Design Data Benefit Layered Design Decomposition?cited by 1
Built on by
Not cited yet.