mlsanityData to 14 Oct 2026

MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsing

arXiv 2212.13492 ↗First posted 27 Dec 2022cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Text-to-SQL semantic parsing is an important NLP task, which greatly facilitates the interaction between users and the database and becomes the key component in many human-computer interaction systems. Much recent progress in text-to-SQL has been driven by large-scale datasets, but most of them are centered on English. In this work, we present MultiSpider, the largest multilingual text-to-SQL dataset which covers seven languages (English, German, French, Spanish, Japanese, Chinese, and Vietnamese). Upon MultiSpider, we further identify the lexical and structural challenges of text-to-SQL (caused by specific language properties and dialect sayings) and their intensity across different languages. Experimental results under three typical settings (zero-shot, monolingual and multilingual) reveal a 6.1% absolute drop in accuracy in non-English languages. Qualitative and quantitative analyses are conducted to understand the reason for the performance drop of each language. Besides the dataset, we also propose a simple schema augmentation framework SAVe (Schema-Augmentation-with-Verification), which significantly boosts the overall performance by about 1.8% and closes the 29.5% performance gap across languages.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 26 of its 32 references that are in the map. First posted on arXiv 27 Dec 2022.

Builds on

26 of 32 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  6. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  7. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  8. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 428
  9. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  10. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  11. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  12. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 185
14 more
  1. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
  2. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 163
  3. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 158
  4. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 149
  5. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 144
  6. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  7. 2018TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generationcited by 88
  8. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 83
  9. 2020A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamesecited by 36
  10. 2019A Pilot Study for Chinese SQL Semantic Parsingcited by 33
  11. 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
  12. 2021mRAT-SQL+GAP:A Portuguese Text-to-SQL Transformercited by 15
  13. 2020Bootstrapping a Crosslingual Semantic Parsercited by 13
  14. 2018Decoupling Structure and Lexicon for Zero-Shot Semantic Parsingcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 8; 2026: 5.

22 papers.

Sort:

By area:

  1. 2024Natural Language Processing for Dialects of a Language: A Surveycited by 40
  2. 2025Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challengescited by 10
  3. 2025Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agentscited by 8
  4. 2024StatBot.Swiss: Bilingual Open Data Exploration in Natural Languagecited by 6
  5. 2024MoMQ: Mixture-of-Experts Enhances Multi-Dialect Query Generation across Relational and Non-Relational Databasescited by 6
  6. 2023XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representationscited by 5
  7. 2024Ar-Spider: Text-to-SQL in Arabiccited by 5
  8. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  9. 2023A Multilingual Translator to SQL with Database Schema Pruning to Improve Self-Attentioncited by 3
  10. 2025Text2Cypher Across Languages: Evaluating and Finetuning LLMscited by 3
  11. 2024Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variabilitycited by 2
  12. 2024PromptMind Team at EHRSQL-2024: Improving Reliability of SQL Generation using Ensemble LLMscited by 2
10 more
  1. 2025MULTITAT: Benchmarking Multilingual Table-and-Text Question Answeringcited by 2
  2. 2026IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languagescited by 1
  3. 2024AraSpider: Democratizing Arabic-to-SQLcited by 0
  4. 2025Multilingual Datasets for Custom Input Extraction and Explanation Requests Parsing in Conversational XAI Systemscited by 0
  5. 2025Bridging the Semantic Gap: Contrastive Rewards for Multilingual Text-to-SQL with GRPOcited by 0
  6. 2025Prompt Engineering Techniques for Context-dependent Text-to-SQL in Arabiccited by 0
  7. 2026Incremental Multilingual Text2Cypher with Adapter Combinationcited by 0
  8. 2026BIRDTurk: Adaptation of the BIRD Text-to-SQL Dataset to Turkishcited by 0
  9. 2026LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQLcited by 0
  10. 2026Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Modelscited by 0