# StarDrinks: An English and Korean Test Set for SLU Evaluation in a Drink Ordering Scenario

arXiv: https://arxiv.org/abs/2604.26500 · HTML: https://mlsanity.com/paper/2604.26500/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 16 references that are in the map.
- First posted on arXiv 29 Apr 2026.

## Builds on (most cited first, 13 of 13)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [SLURP: A Spoken Language Understanding Resource Package](https://mlsanity.com/paper/2011.13205/) (2020)
- [Speech Model Pre-training for End-to-End Spoken Language Understanding](https://mlsanity.com/paper/1904.03670/) (2019)
- [MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages](https://mlsanity.com/paper/2204.08582/) (2022)
- [Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2010.03546/) (2020)
- [On Robustness and Reliability of Benchmark-Based Evaluation of LLMs](https://mlsanity.com/paper/2509.04013/) (2025)
- [STOP: A dataset for Spoken Task Oriented Semantic Parsing](https://mlsanity.com/paper/2207.10643/) (2022)
- [Rethinking Generative Large Language Model Evaluation for Semantic Comprehension](https://mlsanity.com/paper/2403.07872/) (2024)
- [Continual Test-time Adaptation for End-to-end Speech Recognition on Noisy Speech](https://mlsanity.com/paper/2406.11064/) (2024)
- [Cross-TOP: Zero-Shot Cross-Schema Task-Oriented Parsing](https://mlsanity.com/paper/2206.05352/) (2022)
- [LI-TTA: Language Informed Test-Time Adaptation for Automatic Speech Recognition](https://mlsanity.com/paper/2408.05769/) (2024)

## Abstract (arXiv)

LLMs and speech assistants are increasingly used for task-oriented interactions, yet their evaluation often relies on controlled scenarios that fail to capture the variability and complexity of real user requests. Drink ordering, for example, involves diverse named entities, drink types, sizes, customizations, and brand-specific terminology, as well as spontaneous speech phenomena such as hesitations and self-corrections. To address this gap, we introduce StarDrinks, a test set in English and Korean containing speech utterances features, transcriptions, and annotated slots. Our dataset supports speech-to-slots SLU, transcription-to-slots NLU, and speech-to-transcription ASR evaluation, providing a realistic benchmark for model robustness and generalization in a linguistically rich, real-world task.