mlsanityData to 14 Oct 2026

SpotIt+: Verification-based Text-to-SQL Evaluation with Database Constraints

arXiv 2603.04334 ↗First posted 4 Mar 2026cs.DB cs.AI cs.LOcited by 1

Not on the map. Why?

Abstract (arXiv)

We present SpotIt+, an open-source tool for evaluating Text-to-SQL systems via bounded equivalence verification. Given a generated SQL query and the ground truth, SpotIt+ actively searches for database instances that differentiate the two queries. To ensure that the generated counterexamples reflect practically relevant discrepancies, we introduce a best-effort constraint-mining pipeline that combines rule-based specification mining with LLM-based validation over example databases. Experimental results on the BIRD dataset show that the mined constraints enable SpotIt+ to generate more realistic differentiating databases, while preserving its ability to efficiently uncover numerous discrepancies between generated and gold SQL queries that are missed by standard test-based evaluation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 13 of its 29 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

13 of 29 references.

References · 13

  1. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
  2. 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 169
  3. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  4. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 91
  5. 2024RSL-SQL: Robust Schema Linking in Text-to-SQL Generationcited by 52
  6. 2025Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Searchcited by 34
  7. 2025CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learningcited by 15
  8. 2025Cheaper, Better, Faster, Stronger: Robust Text-to-SQL without Chain-of-Thought or Fine-Tuningcited by 13
  9. 2018Axiomatic Foundations and Algorithms for Deciding Semantic Equivalences of SQL Queriescited by 11
  10. 2016HoTTSQL: Proving Query Rewrites with Univalent SQL Semanticscited by 10
  11. 2025SLM-SQL: An Exploration of Small Language Models for Text-to-SQLcited by 10
  12. 2024VeriEQL: Bounded Equivalence Verification for Complex SQL Queries with Integrity Constraintscited by 8
1 more
  1. 2025SpotIt: Evaluating Text-to-SQL Evaluation with Formal Verificationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Certified Against Which Oracle? Execution Labels Set the Reported Risk of Conformal Abstention for Text-to-SQLcited by 0