Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Not on the map. Why?
Abstract (arXiv)
Cascaded speech-to-text translation (S2TT) systems for low-resource languages can suffer from structural noise, particularly the loss of punctuation during the Automatic Speech Recognition (ASR) phase. This research investigates the impact of such noise on Nepali-to-English translation and proposes an optimized pipeline to mitigate quality degradation. We first establish highly proficient ASR and NMT components: a Wav2Vec2-XLS-R-300m model achieved a state-of-the-art 2.72% CER on OpenSLR-54, and a multi-stage fine-tuned MarianMT model reached a 28.32 BLEU score on the FLORES-200 benchmark. We empirically investigate the influence of punctuation loss, demonstrating that unpunctuated ASR output significantly degrades translation quality, causing a massive 20.7% relative BLEU drop on the FLORES benchmark. To overcome this, we propose and evaluate an intermediate Punctuation Restoration Module (PRM). The final S2TT pipeline was tested across three configurations on a custom dataset. The optimal configuration, which applied the PRM directly to ASR output, achieved a 4.90 BLEU point gain over the direct ASR-to-NMT baseline (BLEU 36.38 vs. 31.48). This improvement was validated by human assessment, which confirmed the optimized pipeline's superior Adequacy (3.673) and Fluency (3.804) with inter-rater reliability (Krippendorff's α ≥ 0.723). This work validates that targeted punctuation restoration is the most effective intervention for mitigating structural noise in the Nepali S2TT pipeline. It establishes an optimized baseline and demonstrates a critical architectural insight for developing cascaded speech translation systems for similar low-resource languages.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 23 references that are in the map. First posted on arXiv 25 Feb 2026.
Builds on
11 of 23 references.
References · 11
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
- 2017Six Challenges for Neural Machine Translationcited by 334
- 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
- 2021Survey of Low-Resource Machine Translationcited by 92
- 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
- 2020Applying Wav2vec2.0 to Speech Recognition in Various Low-resource Languagescited by 30
- 2021Wav2vec-S: Semi-Supervised Pre-Training for Low-Resource ASRcited by 7
- 2024Domain-adaptative Continual Learning for Low-resource Tasks: Evaluation on Nepalicited by 5
- 2025Direct Speech to Speech Translation: A Reviewcited by 4
- 2024Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNetcited by 3
- 2025Pivot Language for Low-Resource Machine Translationcited by 3
Built on by
Not cited yet.