mlsanityData to 14 Oct 2026

Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents

arXiv 2509.24405 ↗First posted 29 Sep 2025cs.CL cs.AI cs.DBcited by 8

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Text-to-SQL enables natural access to databases, yet most benchmarks are English-only, limiting multilingual progress. We introduce MultiSpider 2.0, extending Spider 2.0 to eight languages (English, German, French, Spanish, Portuguese, Japanese, Chinese, Vietnamese). It preserves Spider 2.0's structural difficulty while adding linguistic and dialectal variability, demanding deeper reasoning for complex SQL. On this benchmark, state-of-the-art LLMs (such as DeepSeek-R1 and OpenAI o1) reach only 4% execution accuracy when relying on intrinsic reasoning, versus 60% on MultiSpider 1.0. Therefore, we provide a collaboration-driven language agents baseline that iteratively refines queries, improving accuracy to 15%. These results reveal a substantial multilingual gap and motivate methods that are robust across languages and ready for real-world enterprise deployment. Our benchmark is available at https://github.com/phkhanhtrinh23/MultilingualTexttoSQL.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 44 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

32 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL18 Dec

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2021Program Synthesis with Large Language Modelscited by 3,637
  7. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  8. 2023Teaching Large Language Models to Self-Debugcited by 988
  9. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  10. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  11. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  12. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
19 more
  1. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  2. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  3. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
  4. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 195
  5. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 187
  6. 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 169
  7. 2023C3: Zero-shot Text-to-SQL with ChatGPTcited by 164
  8. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 98
  9. 2018TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generationcited by 88
  10. 2022Natural Language to Code Generation in Interactive Data Science Notebookscited by 87
  11. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 83
  12. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thoughtcited by 65
  13. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  14. 2020A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamesecited by 36
  15. 2024Text2SQL is Not Enough: Unifying AI and Databases with TAGcited by 34
  16. 2019A Pilot Study for Chinese SQL Semantic Parsingcited by 33
  17. 2022MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsingcited by 22
  18. 2021mRAT-SQL+GAP:A Portuguese Text-to-SQL Transformercited by 15
  19. 2024LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Trainingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026An Efficient and Effective Evaluator for Text2SQL Models on Unseen and Unlabeled Datacited by 2
  2. 2026IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languagescited by 1
  3. 2026Incremental Multilingual Text2Cypher with Adapter Combinationcited by 0
  4. 2026Empowering Contrastive Federated Sequential Recommendation with LLMscited by 0
  5. 2026Natural Language Interfaces for Spatial and Temporal Databases: A Comprehensive Overview of Methods, Taxonomy, and Future Directionscited by 0
  6. 2026FINER-SQL: Boosting Small Language Models for Text-to-SQLcited by 0
  7. 2026Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflowscited by 0
  8. 2026An Efficient and Effective Agentic Group Shilling Attack on Recommender Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025BAPPA: Benchmarking Agents, Plans, and Pipelines for Automated Text-to-SQL Generation
  2. 2025MultiTEND: A Multilingual Benchmark for Natural Language to NoSQL Query Translation
  3. 2026AgentNLQ: A General-Purpose Agent for Natural Language to SQL
  4. 2025OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment
  5. 2026BIRDTurk: Adaptation of the BIRD Text-to-SQL Dataset to Turkish
  6. 2024Lucy: Think and Reason to Solve Text-to-SQL
  7. 2025A Multi-agent Text2SQL Framework using Small Language Models and Execution Feedback
  8. 2024ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL