mlsanityData to 7 Oct 2026

StatBot.Swiss: Bilingual Open Data Exploration in Natural Language

arXiv 2406.03170 ↗First posted 5 Jun 2024cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

The potential for improvements brought by Large Language Models (LLMs) in Text-to-SQL systems is mostly assessed on monolingual English datasets. However, LLMs' performance for other languages remains vastly unexplored. In this work, we release the StatBot.Swiss dataset, the first bilingual benchmark for evaluating Text-to-SQL systems based on real-world applications. The StatBot.Swiss dataset contains 455 natural language/SQL-pairs over 35 big databases with varying level of complexity for both English and German. We evaluate the performance of state-of-the-art LLMs such as GPT-3.5-Turbo and mixtral-8x7b-instruct for the Text-to-SQL translation task using an in-context learning approach. Our experimental analysis illustrates that current LLMs struggle to generalize well in generating SQL queries on our novel bilingual dataset.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

19 of 33 references.

References · 19

  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  2. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  3. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  4. 2023Teaching Large Language Models to Self-Debugcited by 963
  5. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  6. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  7. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  8. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  9. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  10. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  11. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 118
  12. 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 105
7 more
  1. 2023How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settingscited by 57
  2. 2023ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systemscited by 42
  3. 2020A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamesecited by 36
  4. 2022MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsingcited by 22
  5. 2022Understanding Translationese in Cross-Lingual Summarizationcited by 14
  6. 2023Data Ambiguity Strikes Back: How Documentation Improves GPT's Text-to-SQLcited by 11
  7. 2020A Methodology for Creating Question Answering Corpora Using Inverse Data Annotationcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 3.

6 papers.

  1. 2024SM3-Text-to-Query: Synthetic Multi-Model Medical Text-to-Query Benchmarkcited by 10
  2. 2024Multi-Modal Data Exploration via Language Agentscited by 7
  3. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  4. 2024ETM: Modern Insights into Perspective on Text-to-SQL Evaluation in the Age of Large Language Modelscited by 3
  5. 2025Text2Cypher Across Languages: Evaluating and Finetuning LLMscited by 3
  6. 2025Query Carefully: Detecting the Unanswerables in Text-to-SQL Taskscited by 0