# Granary: Speech Recognition and Translation Dataset in 25 European Languages

arXiv: https://arxiv.org/abs/2505.13404 · HTML: https://mlsanity.com/paper/2505.13404/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 20 of 20)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio](https://mlsanity.com/paper/2106.06909/) (2021)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://mlsanity.com/paper/2305.05084/) (2023)
- [EuroLLM: Multilingual Language Models for Europe](https://mlsanity.com/paper/2409.16235/) (2024)
- [A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models](https://mlsanity.com/paper/2309.11674/) (2023)
- [YODAS: Youtube-Oriented Dataset for Audio and Speech](https://mlsanity.com/paper/2406.00899/) (2024)
- [Lhotse: a speech data representation library for the modern deep learning ecosystem](https://mlsanity.com/paper/2110.12561/) (2021)
- [There's no Data Like Better Data: Using QE Metrics for MT Data Filtering](https://mlsanity.com/paper/2311.05350/) (2023)
- [MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages](https://mlsanity.com/paper/2410.01036/) (2024)
- [Introducing the NewsPaLM MBR and QE Dataset: LLM-Generated High-Quality Parallel Data Outperforms Traditional Web-Crawled Data](https://mlsanity.com/paper/2408.06537/) (2024)
- [Longer is (Not Necessarily) Stronger: Punctuated Long-Sequence Training for Enhanced Speech Recognition and Translation](https://mlsanity.com/paper/2409.05601/) (2024)
- [PyMarian: Fast Neural Machine Translation and Evaluation in Python](https://mlsanity.com/paper/2408.11853/) (2024)

## Built on by (most cited first, 22 of 22)

- [OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models](https://mlsanity.com/paper/2604.00688/) (2026)
- [Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music](https://mlsanity.com/paper/2604.10905/) (2026)
- [Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens](https://mlsanity.com/paper/2509.20060/) (2025)
- [Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs](https://mlsanity.com/paper/2512.16378/) (2025)
- [The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning](https://mlsanity.com/paper/2603.17837/) (2026)
- [X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning](https://mlsanity.com/paper/2605.05611/) (2026)
- [Transcription Policy as a Latent Variable: Activating Controllable Verbatim ASR with Word-Level Timing](https://mlsanity.com/paper/2607.18934/) (2026)
- [Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs](https://mlsanity.com/paper/2604.08003/) (2026)
- [Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularization](https://mlsanity.com/paper/2604.19079/) (2026)
- [WorldSpeech: A Multilingual Speech Corpus from Around the World](https://mlsanity.com/paper/2605.09167/) (2026)
- [Enabling Streaming User Transcription in Full-Duplex Speech-to-Speech Models](https://mlsanity.com/paper/2609.15759/) (2026)
- [Leveraging Multiple Speech Enhancers for Non-Intrusive Intelligibility Prediction for Hearing-Impaired Listeners](https://mlsanity.com/paper/2509.16979/) (2025)
- [Scalable Multilingual Multimodal Machine Translation with Speech-Text Fusion](https://mlsanity.com/paper/2602.21646/) (2026)
- [Identification and Anonymization of Named Entities in Unstructured Information Sources for Use in Social Engineering Detection](https://mlsanity.com/paper/2604.09016/) (2026)
- [AfriVox-v2: A Domain-Verticalized Benchmark for In-the-Wild African Speech Recognition](https://mlsanity.com/paper/2605.03590/) (2026)
- [SCRIBE: Diagnostic Evaluation and Rich Transcription Models for Indic ASR](https://mlsanity.com/paper/2605.20712/) (2026)
- [TRADE: Transducer-Augmented Decoder for Speech LLM](https://mlsanity.com/paper/2606.08486/) (2026)
- [Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR](https://mlsanity.com/paper/2606.24169/) (2026)
- [ZONOS2 Technical Report](https://mlsanity.com/paper/2606.24320/) (2026)
- [Comparing Human and Automatic Recognition of Dutch Dysarthric Continuous Speech: A Case Study](https://mlsanity.com/paper/2606.30237/) (2026)
- [How to Reduce Whisper Hallucination](https://mlsanity.com/paper/2609.32560/) (2026)
- [CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition](https://mlsanity.com/paper/2610.19134/) (2026)

## Abstract (arXiv)

Multi-task and multilingual approaches benefit large models, yet speech processing for low-resource languages remains underexplored due to data scarcity. To address this, we present Granary, a large-scale collection of speech datasets for recognition and translation across 25 European languages. This is the first open-source effort at this scale for both transcription and translation. We enhance data quality using a pseudo-labeling pipeline with segmentation, two-pass inference, hallucination filtering, and punctuation restoration. We further generate translation pairs from pseudo-labeled transcriptions using EuroLLM, followed by a data filtration pipeline. Designed for efficiency, our pipeline processes vast amount of data within hours. We assess models trained on processed data by comparing their performance on previously curated datasets for both high- and low-resource languages. Our findings show that these models achieve similar performance using approx. 50% less data. Dataset will be made available at https://hf.co/datasets/nvidia/Granary