SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps
Not on the map. Why?
Abstract (arXiv)
We introduce SQLSpace, a human-interpretable, generalizable, compact representation for text-to-SQL examples derived with minimal human intervention. We demonstrate the utility of these representations in evaluation with three use cases: (i) closely comparing and contrasting the composition of popular text-to-SQL benchmarks to identify unique dimensions of examples they evaluate, (ii) understanding model performance at a granular level beyond overall accuracy scores, and (iii) improving model performance through targeted query rewriting based on learned correctness estimation. We show that SQLSpace enables analysis that would be difficult with raw examples alone: it reveals compositional differences between benchmarks, exposes performance patterns obscured by accuracy alone, and supports modeling of query success.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 31 Oct 2025.
Builds on
26 of 42 references.
References · 26
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
- 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
- 2018Annotation Artifacts in Natural Language Inference Datacited by 671
- 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
- 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
- 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 185
- 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 144
- 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
- 2024Granite Code Models: A Family of Open Foundation Models for Code Intelligencecited by 117
14 more
- 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
- 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 106
- 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
- 2018Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervisedcited by 75
- 2020On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queriescited by 60
- 2023Goal-Driven Explainable Clustering via Language Descriptionscited by 58
- 2024Synthesizing Text-to-SQL Data from Weak and Strong LLMscited by 54
- 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 53
- 2021Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Datacited by 43
- 2024AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queriescited by 35
- 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
- 2023UNITE: A Unified Benchmark for Text-to-SQL Evaluationcited by 14
- 2018An End-to-end Neural Natural Language Interface for Databasescited by 5
- 2024How often are errors in natural language reasoning due to paraphrastic variability?cited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.