Taming SQL Complexity: LLM-Based Equivalence Evaluation for Text-to-SQL
Not on the map. Why?
Abstract (arXiv)
The rise of Large Language Models (LLMs) has significantly advanced Text-to-SQL (NL2SQL) systems, yet evaluating the semantic equivalence of generated SQL remains a challenge, especially given ambiguous user queries and multiple valid SQL interpretations. This paper explores using LLMs to assess both semantic and a more practical "weak" semantic equivalence. We analyze common patterns of SQL equivalence and inequivalence, discuss challenges in LLM-based evaluation.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 14 of its 25 references that are in the map. First posted on arXiv 11 Jun 2025.
Builds on
14 of 25 references.
References · 14
- 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
- 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
- 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
- 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 164
- 2023Calibrating LLM-Based Evaluatorcited by 53
- 2020Hybrid Ranking Network for Text-to-SQLcited by 33
- 2024HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decompositioncited by 33
- 2024Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teachingcited by 17
- 2024SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Mergingcited by 16
- 2023LLM-SQL-Solver: Can LLMs Determine SQL Equivalence?cited by 10
- 2024SQL-Encoder: Improving NL2SQL In-Context Learning Through a Context-Aware Encodercited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.