mlsanityData to 14 Oct 2026

Analyzing ASR pretraining for low-resource speech-to-text translation

arXiv 1910.10762 ↗First posted 23 Oct 2019cs.CL eess.AScited by 27

Not on the map. Why?

Abstract (arXiv)

Previous work has shown that for low-resource source languages, automatic speech-to-text translation (AST) can be improved by pretraining an end-to-end model on automatic speech recognition (ASR) data from a high-resource language. However, it is not clear what factors --e.g., language relatedness or size of the pretraining data-- yield the biggest improvements, or whether pretraining can be effectively combined with other methods such as data augmentation. Here, we experiment with pretraining on datasets of varying sizes, including languages related and unrelated to the AST source language. We find that the best predictor of final AST performance is the word error rate of the pretrained ASR model, and that differences in ASR/AST performance correlate with how phonetic information is encoded in the later RNN layers of our model. We also show that pretraining and data augmentation yield complementary benefits for AST.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 4; 2023: 8; 2024: 2; 2025: 4; 2026: 1.

27 papers.

Sort:

By area:

  1. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  2. 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
  3. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  4. 2022Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentationcited by 44
  5. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 40
  6. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  7. 2021Learning When to Translate for Streaming Speechcited by 30
  8. 2023CMOT: Cross-modal Mixup via Optimal Transport for Speech Translationcited by 22
  9. 2022Domain Generalization for Activity Recognition via Adaptive Feature Fusioncited by 16
  10. 2023Understanding and Bridging the Modality Gap for Speech Translationcited by 15
  11. 2023Back Translation for Speech-to-text Translation Without Transcriptscited by 9
  12. 2023ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translationcited by 9
15 more
  1. 2023Tuning Large language model for End-to-end Speech Translationcited by 9
  2. 2022When Is TTS Augmentation Through a Pivot Language Useful?cited by 8
  3. 2023Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learningcited by 7
  4. 2023Distributional Data Augmentation Methods for Low Resource Languagecited by 4
  5. 2025When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translationcited by 4
  6. 2021The Volctrans Neural Speech Translation System for IWSLT 2021cited by 3
  7. 2023Strategies for improving low resource speech to text translation relying on pre-trained ASR modelscited by 3
  8. 2021Phoneme Recognition through Fine Tuning of Phonetic Representations: a Case Study on Luhya Language Varietiescited by 2
  9. 2024LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Modelscited by 2
  10. 2025End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Datacited by 1
  11. 2025MLMA: Towards Multilingual ASR With Mamba-based Architecturescited by 1
  12. 2021Speech Representations and Phoneme Classification for Preserving the Endangered Language of Ladincited by 0
  13. 2024LIMBA: An Open-Source Framework for the Preservation and Valorization of Low-Resource Languages using Generative Modelscited by 0
  14. 2025AdaST: Dynamically Adapting Encoder States in the Decoder for End-to-End Speech-to-Text Translationcited by 0
  15. 2026Convex Low-resource Accent-Robust Language Detection in Speech Recognitioncited by 0