# Analyzing ASR pretraining for low-resource speech-to-text translation

arXiv: https://arxiv.org/abs/1910.10762 · HTML: https://mlsanity.com/paper/1910.10762/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 27 of 27)

- [STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation](https://mlsanity.com/paper/2203.10426/) (2022)
- [Learning Shared Semantic Space for Speech-to-Text Translation](https://mlsanity.com/paper/2105.03095/) (2021)
- [Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders](https://mlsanity.com/paper/2105.05752/) (2021)
- [Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation](https://mlsanity.com/paper/2204.02967/) (2022)
- [RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformer](https://mlsanity.com/paper/2106.04833/) (2021)
- [Large-Scale Self- and Semi-Supervised Learning for Speech Translation](https://mlsanity.com/paper/2104.06678/) (2021)
- [Learning When to Translate for Streaming Speech](https://mlsanity.com/paper/2109.07368/) (2021)
- [CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation](https://mlsanity.com/paper/2305.14635/) (2023)
- [Domain Generalization for Activity Recognition via Adaptive Feature Fusion](https://mlsanity.com/paper/2207.11221/) (2022)
- [Understanding and Bridging the Modality Gap for Speech Translation](https://mlsanity.com/paper/2305.08706/) (2023)
- [Back Translation for Speech-to-text Translation Without Transcripts](https://mlsanity.com/paper/2305.08709/) (2023)
- [ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/2305.14838/) (2023)
- [Tuning Large language model for End-to-end Speech Translation](https://mlsanity.com/paper/2310.02050/) (2023)
- [When Is TTS Augmentation Through a Pivot Language Useful?](https://mlsanity.com/paper/2207.09889/) (2022)
- [Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning](https://mlsanity.com/paper/2304.10309/) (2023)
- [Distributional Data Augmentation Methods for Low Resource Language](https://mlsanity.com/paper/2309.04862/) (2023)
- [When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation](https://mlsanity.com/paper/2502.00377/) (2025)
- [The Volctrans Neural Speech Translation System for IWSLT 2021](https://mlsanity.com/paper/2105.07319/) (2021)
- [Strategies for improving low resource speech to text translation relying on pre-trained ASR models](https://mlsanity.com/paper/2306.00208/) (2023)
- [Phoneme Recognition through Fine Tuning of Phonetic Representations: a Case Study on Luhya Language Varieties](https://mlsanity.com/paper/2104.01624/) (2021)
- [LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Models](https://mlsanity.com/paper/2411.00294/) (2024)
- [End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Data](https://mlsanity.com/paper/2506.16251/) (2025)
- [MLMA: Towards Multilingual ASR With Mamba-based Architectures](https://mlsanity.com/paper/2510.18684/) (2025)
- [Speech Representations and Phoneme Classification for Preserving the Endangered Language of Ladin](https://mlsanity.com/paper/2108.12531/) (2021)
- [LIMBA: An Open-Source Framework for the Preservation and Valorization of Low-Resource Languages using Generative Models](https://mlsanity.com/paper/2411.13453/) (2024)
- [AdaST: Dynamically Adapting Encoder States in the Decoder for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/2503.14185/) (2025)
- [Convex Low-resource Accent-Robust Language Detection in Speech Recognition](https://mlsanity.com/paper/2605.23235/) (2026)

## Abstract (arXiv)

Previous work has shown that for low-resource source languages, automatic speech-to-text translation (AST) can be improved by pretraining an end-to-end model on automatic speech recognition (ASR) data from a high-resource language. However, it is not clear what factors --e.g., language relatedness or size of the pretraining data-- yield the biggest improvements, or whether pretraining can be effectively combined with other methods such as data augmentation. Here, we experiment with pretraining on datasets of varying sizes, including languages related and unrelated to the AST source language. We find that the best predictor of final AST performance is the word error rate of the pretrained ASR model, and that differences in ASR/AST performance correlate with how phonetic information is encoded in the later RNN layers of our model. We also show that pretraining and data augmentation yield complementary benefits for AST.