mlsanityData to 7 Oct 2026

SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging

arXiv 2408.12733 ↗First posted 22 Aug 2024cs.AI cs.CL cs.DBcited by 16

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

Recent advances in Text-to-SQL have largely focused on the SQLite dialect, neglecting the diverse landscape of SQL dialects like BigQuery and PostgreSQL. This limitation is due to the diversity in SQL syntaxes and functions, along with the high cost of collecting and curating SQL-specific training data. To address this, we introduce SQL-GEN, a framework for generating high-quality synthetic training data for any SQL dialect, guided by readily available dialect-specific tutorials. SQL-GEN significantly improves cross-dialect Text-to-SQL performance, boosting execution accuracy by up to 20% over existing methods. This performance gain narrows the gap with models trained on large-scale human-annotated data. Furthermore, combining synthetic data from SQL-GEN with human-annotated data yields additional improvements of up to 5.6%. To unify multi-dialect capabilities within a single model, we propose a novel Mixture-of-Experts (MoE) initialization that leverages the shared knowledge across dialects. Our approach merges self-attention layers from dialect-specific models and initializes expert gates using dialect-specific keywords. This leads to a versatile model optimized for multiple SQL dialects, outperforming single-dialect models and significantly enhancing overall performance.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 38 of its 57 references that are in the map. First posted on arXiv 22 Aug 2024.

Builds on

38 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Mixtral of Experts8 Jan

Other references · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  5. 2022Emergent Abilities of Large Language Modelscited by 2,483
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  7. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  8. 2022Editing Models with Task Arithmeticcited by 1,076
  9. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  10. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  11. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  12. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
25 more
  1. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  2. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  3. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  4. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  5. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  6. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 189
  7. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  8. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  9. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  10. 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 164
  11. 2022A Review of Sparse Expert Models in Deep Learningcited by 147
  12. 2023Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Maskscited by 130
  13. 2017Learning a Neural Semantic Parser from User Feedbackcited by 98
  14. 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
  15. 2024Merging Multi-Task Models via Weight-Ensembling Mixture of Expertscited by 85
  16. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  17. 2024Synthesizing Text-to-SQL Data from Weak and Strong LLMscited by 53
  18. 2023SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)cited by 42
  19. 2021Learning to Synthesize Data for Semantic Parsingcited by 41
  20. 2021Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsingcited by 32
  21. 2023Evaluating Cross-Domain Text-to-SQL Models and Benchmarkscited by 32
  22. 2018Question Generation from SQL Queries Improves Neural Semantic Parsingcited by 29
  23. 2022Importance of Synthesizing High-quality Data for Text-to-SQL Parsingcited by 27
  24. 2023SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Datacited by 16
  25. 2024Translating between SQL Dialects for Cloud Migrationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 9; 2026: 4.

16 papers.

Sort:

By area:

  1. 2024CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQLcited by 132
  2. 2024From Natural Language to SQL: Review of LLM-based Text-to-SQL Systemscited by 15
  3. 2024Cultural Palette: Pluralising Culture Alignment via Multi-agent Palettecited by 14
  4. 2025ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialectscited by 11
  5. 2025SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example Selection for Text-to-SQLcited by 4
  6. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  7. 2025SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probescited by 2
  8. 2025Taming SQL Complexity: LLM-Based Equivalence Evaluation for Text-to-SQLcited by 2
  9. 2025AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generationcited by 2
  10. 2025AGRO-SQL: Agentic Group-Relative Optimization with High-Fidelity Data Synthesiscited by 2
  11. 2026CLARITY: A Framework and Benchmark for Conversational Language Ambiguity and Unanswerability in Interactive NL2SQL Systemscited by 2
  12. 2026PolySQL: Scaling Text-to-SQL Evaluation Across SQL Dialects via Automated Backend Isomorphismcited by 1
4 more
  1. 2025Distill-C: Enhanced NL2SQL via Distilled Customization with LLMscited by 0
  2. 2025Do LLMs Align with My Task? Evaluating Text-to-SQL via Dataset Alignmentcited by 0
  3. 2026Knowledge Distillation for Low-Resource Open-source Text-to-SQL Modelcited by 0
  4. 2026Closing the Cross-Dialect Gap: Query Plans as a Portable Interface in Text-to-SQLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SING-SQL: A Synthetic Data Generation Framework for In-Domain Text-to-SQL Translation
  2. 2024MoMQ: Mixture-of-Experts Enhances Multi-Dialect Query Generation across Relational and Non-Relational Databases
  3. 2025Text2SQL-Flow: A Robust SQL-Aware Data Augmentation Framework for Text-to-SQL
  4. 2026Residual Skill Optimization for Text-to-SQL Ensembles
  5. 2025Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents
  6. 2024ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL
  7. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale
  8. 2025Thinkquel: A Model Dedicated to Text-to-dbt Using Synthetic Data and a Span-Aware Objective