From Tens of Hours to Tens of Thousands: Scaling Back-Translation for Speech Recognition
Not on the map. Why?
Abstract (arXiv)
Recent advances in Automatic Speech Recognition (ASR) have been largely fueled by massive speech corpora. However, extending coverage to diverse languages with limited resources remains a formidable challenge. This paper introduces Speech Back-Translation, a scalable pipeline that improves multilingual ASR models by converting large-scale text corpora into synthetic speech via off-the-shelf text-to-speech (TTS) models. We demonstrate that just tens of hours of real transcribed speech can effectively train TTS models to generate synthetic speech at hundreds of times the original volume while maintaining high quality. To evaluate synthetic speech quality, we develop an intelligibility-based assessment framework and establish clear thresholds for when synthetic data benefits ASR training. Using Speech Back-Translation, we generate more than 500,000 hours of synthetic speech in ten languages and continue pre-training Whisper-large-v3, achieving average transcription error reductions of over 30%. These results highlight the scalability and effectiveness of Speech Back-Translation for enhancing multilingual ASR systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 51 references that are in the map. First posted on arXiv 22 May 2025.
Builds on
41 of 51 references.
References · 41
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
- 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
29 more
- 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
- 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
- 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
- 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
- 2023Scaling Speech Technology to 1,000+ Languagescited by 411
- 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
- 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 327
- 2023Effective Data Augmentation With Diffusion Modelscited by 283
- 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
- 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
- 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
- 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
- 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
- 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
- 2016Dual Learning for Machine Translationcited by 148
- 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
- 2023Scaling Laws of Synthetic Images for Model Training ... for Nowcited by 108
- 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 99
- 2024Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognitioncited by 63
- 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 50
- 2024G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translationcited by 30
- 2023Speech Translation with Large Language Models: An Industrial Practicecited by 28
- 2024Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoningcited by 26
- 2024ECAPA2: A Hybrid Neural Network Architecture and Training Strategy for Robust Speaker Embeddingscited by 22
- 2021Voice Conversion Can Improve ASR in Very Low-Resource Settingscited by 14
- 2024Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agentcited by 14
- 2023Learning Multi-Step Reasoning by Solving Arithmetic Taskscited by 12
- 2024Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gapcited by 12
- 2022Differentiable Data Augmentation for Contrastive Sentence Representation Learningcited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.