# MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsing

arXiv: https://arxiv.org/abs/2212.13492 · HTML: https://mlsanity.com/paper/2212.13492/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 27 Dec 2022.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://mlsanity.com/paper/2109.05093/) (2021)
- [Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation](https://mlsanity.com/paper/1905.08205/) (2019)
- [CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases](https://mlsanity.com/paper/1909.05378/) (2019)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark](https://mlsanity.com/paper/2008.09335/) (2020)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [SParC: Cross-Domain Semantic Parsing in Context](https://mlsanity.com/paper/1906.02285/) (2019)
- [Towards Robustness of Text-to-SQL Models against Synonym Substitution](https://mlsanity.com/paper/2106.01065/) (2021)
- [TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation](https://mlsanity.com/paper/1810.02720/) (2018)
- [End-to-End Slot Alignment and Recognition for Cross-Lingual NLU](https://mlsanity.com/paper/2004.14353/) (2020)
- [A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese](https://mlsanity.com/paper/2010.01891/) (2020)
- [A Pilot Study for Chinese SQL Semantic Parsing](https://mlsanity.com/paper/1909.13293/) (2019)
- [Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation](https://mlsanity.com/paper/2212.09994/) (2022)
- [mRAT-SQL+GAP:A Portuguese Text-to-SQL Transformer](https://mlsanity.com/paper/2110.03546/) (2021)
- [Bootstrapping a Crosslingual Semantic Parser](https://mlsanity.com/paper/2004.02585/) (2020)
- [Decoupling Structure and Lexicon for Zero-Shot Semantic Parsing](https://mlsanity.com/paper/1804.07918/) (2018)

## Built on by (most cited first, 22 of 22)

- [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024)
- [Tabular Data Understanding with LLMs: A Survey of Recent Advances and Challenges](https://mlsanity.com/paper/2508.00217/) (2025)
- [Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents](https://mlsanity.com/paper/2509.24405/) (2025)
- [StatBot.Swiss: Bilingual Open Data Exploration in Natural Language](https://mlsanity.com/paper/2406.03170/) (2024)
- [MoMQ: Mixture-of-Experts Enhances Multi-Dialect Query Generation across Relational and Non-Relational Databases](https://mlsanity.com/paper/2410.18406/) (2024)
- [XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations](https://mlsanity.com/paper/2306.04085/) (2023)
- [Ar-Spider: Text-to-SQL in Arabic](https://mlsanity.com/paper/2402.15012/) (2024)
- [Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities](https://mlsanity.com/paper/2505.23838/) (2025)
- [A Multilingual Translator to SQL with Database Schema Pruning to Improve Self-Attention](https://mlsanity.com/paper/2306.14256/) (2023)
- [Text2Cypher Across Languages: Evaluating and Finetuning LLMs](https://mlsanity.com/paper/2506.21445/) (2025)
- [Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variability](https://mlsanity.com/paper/2404.07135/) (2024)
- [PromptMind Team at EHRSQL-2024: Improving Reliability of SQL Generation using Ensemble LLMs](https://mlsanity.com/paper/2405.08839/) (2024)
- [MULTITAT: Benchmarking Multilingual Table-and-Text Question Answering](https://mlsanity.com/paper/2502.17253/) (2025)
- [IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages](https://mlsanity.com/paper/2604.13686/) (2026)
- [AraSpider: Democratizing Arabic-to-SQL](https://mlsanity.com/paper/2402.07448/) (2024)
- [Multilingual Datasets for Custom Input Extraction and Explanation Requests Parsing in Conversational XAI Systems](https://mlsanity.com/paper/2508.14982/) (2025)
- [Bridging the Semantic Gap: Contrastive Rewards for Multilingual Text-to-SQL with GRPO](https://mlsanity.com/paper/2510.13827/) (2025)
- [Prompt Engineering Techniques for Context-dependent Text-to-SQL in Arabic](https://mlsanity.com/paper/2511.20677/) (2025)
- [Incremental Multilingual Text2Cypher with Adapter Combination](https://mlsanity.com/paper/2601.16097/) (2026)
- [BIRDTurk: Adaptation of the BIRD Text-to-SQL Dataset to Turkish](https://mlsanity.com/paper/2602.03633/) (2026)
- [LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQL](https://mlsanity.com/paper/2604.18254/) (2026)
- [Beyond English: Uncovering the Multilingual Gap in Vision-Language-Action Models](https://mlsanity.com/paper/2606.15714/) (2026)

## Abstract (arXiv)

Text-to-SQL semantic parsing is an important NLP task, which greatly facilitates the interaction between users and the database and becomes the key component in many human-computer interaction systems. Much recent progress in text-to-SQL has been driven by large-scale datasets, but most of them are centered on English. In this work, we present MultiSpider, the largest multilingual text-to-SQL dataset which covers seven languages (English, German, French, Spanish, Japanese, Chinese, and Vietnamese). Upon MultiSpider, we further identify the lexical and structural challenges of text-to-SQL (caused by specific language properties and dialect sayings) and their intensity across different languages. Experimental results under three typical settings (zero-shot, monolingual and multilingual) reveal a 6.1% absolute drop in accuracy in non-English languages. Qualitative and quantitative analyses are conducted to understand the reason for the performance drop of each language. Besides the dataset, we also propose a simple schema augmentation framework SAVe (Schema-Augmentation-with-Verification), which significantly boosts the overall performance by about 1.8% and closes the 29.5% performance gap across languages.