EntSQL: A Benchmark for Grounding Text-to-SQL in Long-Context Enterprise Knowledge
Not on the map. Why?
Abstract (arXiv)
Text-to-SQL enables natural language access to databases, and recent LLMs have substantially advanced its capabilities. Existing benchmarks such as Spider, BIRD, and Spider~2.0 evaluate schema generalization, large-scale databases, and realistic workflows, but largely overlook enterprise scenarios where SQL generation depends on private business knowledge, such as internal metrics, reporting conventions, and organizational rules. We introduce EntSQL, an enterprise-oriented Text-to-SQL benchmark for evaluating long-context grounding over proprietary business documents. EntSQL contains 1,066 aligned Chinese-English semantic examples across five business domains, with most examples requiring domain knowledge beyond the question and schema and involving complex SQL structures. On English inputs, the best evaluated system reaches only 15.9% when long-form documents are provided, highlighting the difficulty of grounding SQL generation in enterprise knowledge.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 2 Jun 2026.
Builds on
20 of 29 references.
References · 20
- 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
- 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
- 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
- 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
- 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
- 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 223
- 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
- 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
- 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
- 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 164
8 more
- 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQLcited by 158
- 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
- 2017SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learningcited by 135
- 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
- 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 105
- 2020SmBoP: Semi-autoregressive Bottom-up Semantic Parsingcited by 97
- 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
- 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 50
Built on by
Citing papers per year: 2026: 1.
1 paper.