# StatBot.Swiss: Bilingual Open Data Exploration in Natural Language

arXiv: https://arxiv.org/abs/2406.03170 · HTML: https://mlsanity.com/paper/2406.03170/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://mlsanity.com/paper/2109.05093/) (2021)
- [A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability](https://mlsanity.com/paper/2303.13547/) (2023)
- [KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers](https://mlsanity.com/paper/2106.11455/) (2021)
- [How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settings](https://mlsanity.com/paper/2305.11853/) (2023)
- [ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems](https://mlsanity.com/paper/2306.04743/) (2023)
- [A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese](https://mlsanity.com/paper/2010.01891/) (2020)
- [MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsing](https://mlsanity.com/paper/2212.13492/) (2022)
- [Understanding Translationese in Cross-Lingual Summarization](https://mlsanity.com/paper/2212.07220/) (2022)
- [Data Ambiguity Strikes Back: How Documentation Improves GPT's Text-to-SQL](https://mlsanity.com/paper/2310.18742/) (2023)
- [A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation](https://mlsanity.com/paper/2004.07633/) (2020)

## Built on by (most cited first, 6 of 6)

- [SM3-Text-to-Query: Synthetic Multi-Model Medical Text-to-Query Benchmark](https://mlsanity.com/paper/2411.05521/) (2024)
- [Multi-Modal Data Exploration via Language Agents](https://mlsanity.com/paper/2412.18428/) (2024)
- [Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities](https://mlsanity.com/paper/2505.23838/) (2025)
- [ETM: Modern Insights into Perspective on Text-to-SQL Evaluation in the Age of Large Language Models](https://mlsanity.com/paper/2407.07313/) (2024)
- [Text2Cypher Across Languages: Evaluating and Finetuning LLMs](https://mlsanity.com/paper/2506.21445/) (2025)
- [Query Carefully: Detecting the Unanswerables in Text-to-SQL Tasks](https://mlsanity.com/paper/2512.21345/) (2025)

## Abstract (arXiv)

The potential for improvements brought by Large Language Models (LLMs) in Text-to-SQL systems is mostly assessed on monolingual English datasets. However, LLMs' performance for other languages remains vastly unexplored. In this work, we release the StatBot.Swiss dataset, the first bilingual benchmark for evaluating Text-to-SQL systems based on real-world applications. The StatBot.Swiss dataset contains 455 natural language/SQL-pairs over 35 big databases with varying level of complexity for both English and German. We evaluate the performance of state-of-the-art LLMs such as GPT-3.5-Turbo and mixtral-8x7b-instruct for the Text-to-SQL translation task using an in-context learning approach. Our experimental analysis illustrates that current LLMs struggle to generalize well in generating SQL queries on our novel bilingual dataset.