# Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL

arXiv: https://arxiv.org/abs/2010.12634 · HTML: https://mlsanity.com/paper/2010.12634/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 16 of 16)

- [KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers](https://mlsanity.com/paper/2106.11455/) (2021)
- [Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect](https://mlsanity.com/paper/2208.10099/) (2022)
- [EHRSQL: A Practical Text-to-SQL Benchmark for Electronic Health Records](https://mlsanity.com/paper/2301.07695/) (2023)
- [PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries](https://mlsanity.com/paper/2410.11076/) (2024)
- [TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoring](https://mlsanity.com/paper/2403.15879/) (2024)
- [Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Survey](https://mlsanity.com/paper/2208.04415/) (2022)
- [NL2INTERFACE: Interactive Visualization Interface Generation from Natural Language Queries](https://mlsanity.com/paper/2209.08834/) (2022)
- [QDA-SQL: Questions Enhanced Dialogue Augmentation for Multi-Turn Text-to-SQL](https://mlsanity.com/paper/2406.10593/) (2024)
- [Towards Unbiased Evaluation of Detecting Unanswerable Questions in EHRSQL](https://mlsanity.com/paper/2405.01588/) (2024)
- [Evaluating and Enhancing LLMs for Multi-turn Text-to-SQL with Multiple Question Types](https://mlsanity.com/paper/2412.17867/) (2024)
- [Confidence Estimation for Text-to-SQL in Large Language Models](https://mlsanity.com/paper/2508.14056/) (2025)
- [LatentRefusal: Latent-Signal Refusal for Unanswerable Text-to-SQL Queries](https://mlsanity.com/paper/2601.10398/) (2026)
- [SCARE: A Benchmark for SQL Correction and Question Answerability Classification for Reliable EHR Question Answering](https://mlsanity.com/paper/2511.17559/) (2025)
- [Query Carefully: Detecting the Unanswerables in Text-to-SQL Tasks](https://mlsanity.com/paper/2512.21345/) (2025)
- [Natural Language Interfaces for Spatial and Temporal Databases: A Comprehensive Overview of Methods, Taxonomy, and Future Directions](https://mlsanity.com/paper/2603.23375/) (2026)
- [TrustDABench: Benchmarking Reliability and Robustness of LLMs for Structured Data Analysis](https://mlsanity.com/paper/2608.24145/) (2026)

## Abstract (arXiv)

Neural models have achieved significant results on the text-to-SQL task, in which most current work assumes all the input questions are legal and generates a SQL query for any input. However, in the real scenario, users can input any text that may not be able to be answered by a SQL query. In this work, we propose TriageSQL, the first cross-domain text-to-SQL question intention classification benchmark that requires models to distinguish four types of unanswerable questions from answerable questions. The baseline RoBERTa model achieves a 60% F1 score on the test set, demonstrating the need for further improvement on this task. Our dataset is available at https://github.com/chatc/TriageSQL.