mlsanityData to 7 Oct 2026

Business Logic-Driven Text-to-SQL Data Synthesis for Business Intelligence

arXiv 2601.14518 ↗First posted 20 Jan 2026cs.CLcited by 0

On the map: Synthetic data and self-improvement for LLMs → Synthetic reasoning data and bootstrapping

Abstract (arXiv)

Evaluating Text-to-SQL agents in private business intelligence (BI) settings is challenging due to the scarcity of realistic, domain-specific data. While synthetic evaluation data offers a scalable solution, existing generation methods fail to capture business realism--whether questions reflect realistic business logic and workflows. We propose a Business Logic-Driven Data Synthesis framework that generates data grounded in business personas, work scenarios, and workflows. In addition, we improve the data quality by imposing a business reasoning complexity control strategy that diversifies the analytical reasoning steps required to answer the questions. Experiments on a production-scale Salesforce database show that our synthesized data achieves high business realism (98.44%), substantially outperforming OmniSQL (+19.5%) and SQL-Factory (+54.7%), while maintaining strong question-SQL alignment (98.59%). Our synthetic data also reveals that state-of-the-art Text-to-SQL models still have significant performance gaps, achieving only 42.86% execution accuracy on the most complex business queries.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Synthetic reasoning data and bootstrapping, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 10 of its 26 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

10 of 26 references.

References · 10

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  3. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  4. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  5. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 90
  6. 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environmentscited by 69
  7. 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
  8. 2025CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactionscited by 46
  9. 2025SQL-Factory: A Multi-Agent Framework for High-Quality and Large-Scale SQL Generationcited by 5
  10. 2025SQLBarber: A System Leveraging Large Language Models to Generate Customized and Realistic SQL Workloadscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fact-Consistency Evaluation of Text-to-SQL Generation for Business Intelligence Using Exaone 3.5
  2. 2025SING-SQL: A Synthetic Data Generation Framework for In-Domain Text-to-SQL Translation
  3. 2026Business Truth, not SQL Accuracy: A Rule-Gated 7B Analytics Agent Outperforms a Direct-Prompted 32B Baseline
  4. 2025Agent Bain vs. Agent McKinsey: A New Text-to-SQL Benchmark for the Business Domain
  5. 2024BEAVER: An Enterprise Benchmark for Text-to-SQL
  6. 2026APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQL
  7. 2026SemanticAgent: A Semantics-Aware Framework for Text-to-SQL Data Synthesis
  8. 2025BAPPA: Benchmarking Agents, Plans, and Pipelines for Automated Text-to-SQL Generation