mlsanityData to 14 Oct 2026

PiDA: Phonetically-Informed Data Augmentation for Robust Vietnamese Speech Translation

arXiv 2606.12911 ↗First posted 11 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Cascaded speech translation (ST) systems suffer from error propagation when Automatic Speech Recognition (ASR) outputs incorrect transcripts. We present the first systematic categorization of ASR errors for Vietnamese ST, classifying substitution errors by phonetic cause and quantifying their impact on downstream Neural Machine Translation (NMT) performance using Linear Mixed-Effects Modelling. We confirm that most ASR substitution errors arise from phonetic confusions rather than random noise, and that these phonetic errors significantly degrade ST quality. Motivated by this finding, we propose Phonetically-Informed Data Augmentation (PiDA), which generates ASR-like corruptions by substituting words with phonetically similar alternatives using phonetic word embeddings. Fine-tuning on a PiDA-augmented version of FLEURS Vietnamese-English improves translation of erroneous ASR outputs (up to +2.04 BLEU over standard fine-tuning) while also slightly improving clean-text performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 40 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

22 of 40 references.

References · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023Mistral 7Bcited by 3,164
  6. 2017Billion-scale similarity search with GPUscited by 2,497
  7. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  9. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  10. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  11. 2022ByT5 model for massively multilingual grapheme-to-phoneme conversioncited by 49
  12. 2018Towards Robust Neural Machine Translationcited by 35
10 more
  1. 2023XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speechcited by 16
  2. 2019Robust Neural Machine Translation for Clean and Noisy Speech Transcriptscited by 13
  3. 2024PhoWhisper: Automatic Speech Recognition for Vietnamesecited by 13
  4. 2021PhoMT: A High-Quality and Large-Scale Benchmark Dataset for Vietnamese-English Machine Translationcited by 12
  5. 2024MEDSAGE: Enhancing Robustness of Medical Dialogue Summarization to ASR Errors with LLM-generated Synthetic Dialoguescited by 8
  6. 2023PWESuite: Phonetic Word Embeddings and Tasks They Facilitatecited by 6
  7. 2022MTet: Multi-domain Translation for English and Vietnamesecited by 5
  8. 2024Improving Vietnamese-English Medical Machine Translationcited by 5
  9. 2026ViMedCSS: A Vietnamese Medical Code-Switching Speech Dataset & Benchmarkcited by 3
  10. 2025MultiMed-ST: Large-scale Many-to-many Multilingual Medical Speech Translationcited by 2

Built on by

Not cited yet.