mlsanityData to 7 Oct 2026

MoMQ: Mixture-of-Experts Enhances Multi-Dialect Query Generation across Relational and Non-Relational Databases

arXiv 2410.18406 ↗First posted 24 Oct 2024cs.CL cs.AI cs.DBcited by 6

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

The improvement in translating natural language to structured query language (SQL) can be attributed to the advancements in large language models (LLMs). Open-source LLMs, tailored for specific database dialects such as MySQL, have shown great performance. However, cloud service providers are looking for a unified database manager service (e.g., Cosmos DB from Azure, Amazon Aurora from AWS, Lindorm from AlibabaCloud) that can support multiple dialects. This requirement has led to the concept of multi-dialect query generation, which presents challenges to LLMs. These challenges include syntactic differences among dialects and imbalanced data distribution across multiple dialects. To tackle these challenges, we propose MoMQ, a novel Mixture-of-Experts-based multi-dialect query generation framework across both relational and non-relational databases. MoMQ employs a dialect expert group for each dialect and a multi-level routing strategy to handle dialect-specific knowledge, reducing interference during query generation. Additionally, a shared expert group is introduced to address data imbalance, facilitating the transfer of common knowledge from high-resource dialects to low-resource ones. Furthermore, we have developed a high-quality multi-dialect query generation benchmark that covers relational and non-relational databases such as MySQL, PostgreSQL, Cypher for Neo4j, and nGQL for NebulaGraph. Extensive experiments have shown that MoMQ performs effectively and robustly even in resource-imbalanced scenarios.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 23 of its 32 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

23 of 32 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models29 Jan
  2. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints9 Dec
  3. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Experts13 Dec
  4. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding30 Jun
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer23 Jan

Other references · 18

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  6. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  7. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  8. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  9. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  10. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  11. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  12. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
6 more
  1. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  2. 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
  3. 2024PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistencycited by 26
  4. 2022MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsingcited by 22
  5. 2022Nebula Graph: An open source distributed graph databasecited by 8
  6. 2023R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQLcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQLcited by 40
  2. 2025ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialectscited by 11
  3. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  4. 2025SQL-Exchange: Transforming SQL Queries Across Domainscited by 1
  5. 2026PolySQL: Scaling Text-to-SQL Evaluation Across SQL Dialects via Automated Backend Isomorphismcited by 1
  6. 2026Closing the Cross-Dialect Gap: Query Plans as a Portable Interface in Text-to-SQLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging
  2. 2025MultiTEND: A Multilingual Benchmark for Natural Language to NoSQL Query Translation
  3. 2026Optimizing Small Language Models for NL2SQL via Chain-of-Thought Fine-Tuning
  4. 2025LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL
  5. 2025Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents
  6. 2025DeKeyNLU: Enhancing Natural Language to SQL Generation through Task Decomposition and Keyword Extraction
  7. 2025Querying Databases with Function Calling
  8. 2023Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generation