# Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents

arXiv: https://arxiv.org/abs/2509.24405 · HTML: https://mlsanity.com/paper/2509.24405/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 44 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 30 of 32)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows](https://mlsanity.com/paper/2411.07763/) (2024)
- [RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL](https://mlsanity.com/paper/2302.05965/) (2023)
- [CodeS: Towards Building Open-source Language Models for Text-to-SQL](https://mlsanity.com/paper/2402.16347/) (2024)
- [CHESS: Contextual Harnessing for Efficient SQL Synthesis](https://mlsanity.com/paper/2405.16755/) (2024)
- [C3: Zero-shot Text-to-SQL with ChatGPT](https://mlsanity.com/paper/2307.07306/) (2023)
- [MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL](https://mlsanity.com/paper/2312.11242/) (2023)
- [NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System](https://mlsanity.com/paper/1802.08979/) (2018)
- [TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation](https://mlsanity.com/paper/1810.02720/) (2018)
- [Natural Language to Code Generation in Interactive Data Science Notebooks](https://mlsanity.com/paper/2212.09248/) (2022)
- [End-to-End Slot Alignment and Recognition for Cross-Lingual NLU](https://mlsanity.com/paper/2004.14353/) (2020)
- [ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought](https://mlsanity.com/paper/2310.17342/) (2023)
- [Generating Natural Language Proofs with Verifier-Guided Search](https://mlsanity.com/paper/2205.12443/) (2022)
- [A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese](https://mlsanity.com/paper/2010.01891/) (2020)
- [Text2SQL is Not Enough: Unifying AI and Databases with TAG](https://mlsanity.com/paper/2408.14717/) (2024)
- [A Pilot Study for Chinese SQL Semantic Parsing](https://mlsanity.com/paper/1909.13293/) (2019)
- [MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsing](https://mlsanity.com/paper/2212.13492/) (2022)

## Built on by (most cited first, 8 of 8)

- [An Efficient and Effective Evaluator for Text2SQL Models on Unseen and Unlabeled Data](https://mlsanity.com/paper/2603.07841/) (2026)
- [IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages](https://mlsanity.com/paper/2604.13686/) (2026)
- [Incremental Multilingual Text2Cypher with Adapter Combination](https://mlsanity.com/paper/2601.16097/) (2026)
- [Empowering Contrastive Federated Sequential Recommendation with LLMs](https://mlsanity.com/paper/2602.09306/) (2026)
- [Natural Language Interfaces for Spatial and Temporal Databases: A Comprehensive Overview of Methods, Taxonomy, and Future Directions](https://mlsanity.com/paper/2603.23375/) (2026)
- [FINER-SQL: Boosting Small Language Models for Text-to-SQL](https://mlsanity.com/paper/2605.03465/) (2026)
- [Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows](https://mlsanity.com/paper/2607.06229/) (2026)
- [An Efficient and Effective Agentic Group Shilling Attack on Recommender Systems](https://mlsanity.com/paper/2609.09551/) (2026)

## Abstract (arXiv)

Text-to-SQL enables natural access to databases, yet most benchmarks are English-only, limiting multilingual progress. We introduce MultiSpider 2.0, extending Spider 2.0 to eight languages (English, German, French, Spanish, Portuguese, Japanese, Chinese, Vietnamese). It preserves Spider 2.0's structural difficulty while adding linguistic and dialectal variability, demanding deeper reasoning for complex SQL. On this benchmark, state-of-the-art LLMs (such as DeepSeek-R1 and OpenAI o1) reach only 4% execution accuracy when relying on intrinsic reasoning, versus 60% on MultiSpider 1.0. Therefore, we provide a collaboration-driven language agents baseline that iteratively refines queries, improving accuracy to 15%. These results reveal a substantial multilingual gap and motivate methods that are robust across languages and ready for real-world enterprise deployment. Our benchmark is available at https://github.com/phkhanhtrinh23/MultilingualTexttoSQL.