mlsanityData to 7 Oct 2026

Text-to-SQL Domain Adaptation via Human-LLM Collaborative Data Annotation

arXiv 2502.15980 ↗First posted 21 Feb 2025cs.HC cs.AI cs.DBcited by 6

Not on the map. Why?

Abstract (arXiv)

Text-to-SQL models, which parse natural language (NL) questions to executable SQL queries, are increasingly adopted in real-world applications. However, deploying such models in the real world often requires adapting them to the highly specialized database schemas used in specific applications. We find that existing text-to-SQL models experience significant performance drops when applied to new schemas, primarily due to the lack of domain-specific data for fine-tuning. This data scarcity also limits the ability to effectively evaluate model performance in new domains. Continuously obtaining high-quality text-to-SQL data for evolving schemas is prohibitively expensive in real-world scenarios. To bridge this gap, we propose SQLsynth, a human-in-the-loop text-to-SQL data annotation system. SQLsynth streamlines the creation of high-quality text-to-SQL datasets through human-LLM collaboration in a structured workflow. A within-subjects user study comparing SQLsynth with manual annotation and ChatGPT shows that SQLsynth significantly accelerates text-to-SQL data annotation, reduces cognitive load, and produces datasets that are more accurate, natural, and diverse. Our code is available at https://github.com/magic-YuanTian/SQLsynth.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 51 of its 100 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

51 of 100 references.

References · 51

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  7. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  8. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  9. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  10. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  11. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  12. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
38 more
  1. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  2. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  3. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  4. 2020A Brief Review of Domain Adaptationcited by 287
  5. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  6. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  7. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  8. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
  9. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
  10. 2018Improving Text-to-SQL Evaluation Methodologycited by 133
  11. 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
  12. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  13. 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 105
  14. 2022Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectivescited by 98
  15. 2020SmBoP: Semi-autoregressive Bottom-up Semantic Parsingcited by 97
  16. 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
  17. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  18. 2019Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questionscited by 80
  19. 2020Grounded Adaptation for Zero-shot Executable Semantic Parsingcited by 76
  20. 2018SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Taskcited by 74
  21. 2024The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Modelscited by 67
  22. 2020Reinforced active learning for image segmentationcited by 56
  23. 2021Learning to Synthesize Data for Semantic Parsingcited by 41
  24. 2021Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsingcited by 32
  25. 2018SQL-to-Text Generation with Graph-to-Sequence Modelcited by 31
  26. 2018Question Generation from SQL Queries Improves Neural Semantic Parsingcited by 29
  27. 2016Fast Domain Adaptation for Neural Machine Translationcited by 28
  28. 2024Selective Prompt Anchoring for Code Generationcited by 16
  29. 2019Bridging the Semantic Gap with SQL Query Logs in Natural Language Interfaces to Databasescited by 15
  30. 2023FreeAL: Towards Human-Free Active Learning in the Era of Large Language Modelscited by 14
  31. 2024Dubo-SQL: Diverse Retrieval-Augmented Generation and Fine Tuning for Text-to-SQLcited by 12
  32. 2020Semi-Automatic Data Annotation guided by Feature Space Projectioncited by 11
  33. 2024GPTs Are Multilingual Annotators for Sequence Generation Taskscited by 11
  34. 2024Insights into Natural Language Database Query Errors: From Attention Misalignment to User Handling Strategiescited by 11
  35. 2023LLM-SQL-Solver: Can LLMs Determine SQL Equivalence?cited by 10
  36. 2024ARAIDA: Analogical Reasoning-Augmented Interactive Data Annotationcited by 6
  37. 2017Type- and Content-Driven Synthesis of SQL Queries from Natural Languagecited by 5
  38. 2022IRNet: Iterative Refinement Network for Noisy Partial Label Learningcited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2026PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agentscited by 5
  2. 2026TACO: Task-Aware Column Description Generation Using LLMscited by 1
  3. 2025FINCH: Financial Intelligence using Natural language for Contextualized SQL Handlingcited by 0
  4. 2025Adobe Summit Concierge Evaluation with Human in the Loopcited by 0
  5. 2026Bridging Natural Language and Interactive What-If Interfaces via LLM-Generated Declarative Specificationscited by 0
  6. 2026ISEE: Interactive Semantic Enrichment for Database Fieldscited by 0