mlsanityData to 7 Oct 2026

BIRDTurk: Adaptation of the BIRD Text-to-SQL Dataset to Turkish

arXiv 2602.03633 ↗First posted 3 Feb 2026cs.CL cs.AI cs.DBcited by 0

Not on the map. Why?

Abstract (arXiv)

Text-to-SQL systems have achieved strong performance on English benchmarks, yet their behavior in morphologically rich, low-resource languages remains largely unexplored. We introduce BIRDTurk, the first Turkish adaptation of the BIRD benchmark, constructed through a controlled translation pipeline that adapts schema identifiers to Turkish while strictly preserving the logical structure and execution semantics of SQL queries and databases. Translation quality is validated on a sample size determined by the Central Limit Theorem to ensure 95% confidence, achieving 98.15% accuracy on human-evaluated samples. Using BIRDTurk, we evaluate inference-based prompting, agentic multi-stage reasoning, and supervised fine-tuning. Our results reveal that Turkish introduces consistent performance degradation, driven by both structural linguistic divergence and underrepresentation in LLM pretraining, while agentic reasoning demonstrates stronger cross-lingual robustness. Supervised fine-tuning remains challenging for standard multilingual baselines but scales effectively with modern instruction-tuned models. BIRDTurk provides a controlled testbed for cross-lingual Text-to-SQL evaluation under realistic database conditions. We release the training and development splits to support future research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 30 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

15 of 30 references.

References · 15

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  2. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  3. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  4. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  5. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  6. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 313
  7. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 223
  8. 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
  9. 2018Improving Text-to-SQL Evaluation Methodologycited by 133
  10. 2022Impact of Tokenization on Language Models: An Analysis for Turkishcited by 62
  11. 2020A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamesecited by 36
  12. 2019A Pilot Study for Chinese SQL Semantic Parsingcited by 33
3 more
  1. 2022MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsingcited by 22
  2. 2025Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkishcited by 12
  3. 2024Ar-Spider: Text-to-SQL in Arabiccited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents
  2. 2025Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL
  3. 2025SLM-SQL: An Exploration of Small Language Models for Text-to-SQL
  4. 2024Bridging the Bosphorus: Advancing Turkish Large Language Models through Strategies for Low-Resource Language Adaptation and Benchmarking
  5. 2025JudgeSQL: Reasoning over SQL Candidates with Weighted Consensus Tournament
  6. 2025SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs
  7. 2025SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQL
  8. 2025BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactions