An Efficient and Effective Evaluator for Text2SQL Models on Unseen and Unlabeled Data
Not on the map. Why?
Abstract (arXiv)
Recent advances in large language models have strengthened Text2SQL systems that translate natural language questions into database queries. A persistent deployment challenge is to assess a newly trained Text2SQL system on an unseen and unlabeled dataset when no verified answers are available. This situation arises frequently because database content and structure evolve, privacy policies slow manual review, and carefully written SQL labels are costly and time-consuming. Without timely evaluation, organizations cannot approve releases or detect failures early. FusionSQL addresses this gap by working with any Text2SQL models and estimating accuracy without reference labels, allowing teams to measure quality on unseen and unlabeled datasets. It analyzes patterns in the system's own outputs to characterize how the target dataset differs from the material used during training. FusionSQL supports pre-release checks, continuous monitoring of new databases, and detection of quality decline. Experiments across diverse application settings and question types show that FusionSQL closely follows actual accuracy and reliably signals emerging issues. Our code is available at https://github.com/phkhanhtrinh23/FusionSQL.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 64 of its 83 references that are in the map. First posted on arXiv 8 Mar 2026.
Builds on
64 of 83 references.
References · 64
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2024Mixtral of Expertscited by 1,823
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
- 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
- 2018On First-Order Meta-Learning Algorithmscited by 1,018
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
38 more
- 2024Yi: Open Foundation Models by 01.AIcited by 719
- 2024Phi-4 Technical Reportcited by 711
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
- 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
- 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
- 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
- 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
- 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
- 2024InternLM2 Technical Reportcited by 397
- 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
- 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
- 2023Prediction-Powered Inferencecited by 220
- 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
- 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 195
- 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 185
- 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
- 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 169
- 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 145
- 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 144
- 2024CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQLcited by 138
- 2017SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learningcited by 135
- 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
- 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
- 2021Predicting with Confidence on Unseen Distributionscited by 116
- 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 106
- 2020SmBoP: Semi-autoregressive Bottom-up Semantic Parsingcited by 97
- 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 91
- 2018TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generationcited by 81
- 2019Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questionscited by 80
- 2024The Dawn of Natural Language to SQL: Are We Fully Ready?cited by 79
- 2022PromptBERT: Improving BERT Sentence Embeddings with Promptscited by 75
- 2018SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Taskcited by 74
- 2024AutoEval Done Right: Using Synthetic Data for Model Evaluationcited by 57
- 2017Multilevel Clustering via Wasserstein Meanscited by 49
- 2023EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Recordscited by 45
- 2023ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systemscited by 45
- 2025XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQLcited by 40
- 2025Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Searchcited by 34
50 most cited of 64 All 64 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.