mlsanityData to 7 Oct 2026

Romanization Encoding For Multilingual ASR

arXiv 2407.04368 ↗First posted 5 Jul 2024cs.CL cs.SD eess.AScited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce romanization encoding for script-heavy languages to optimize multilingual and code-switching Automatic Speech Recognition (ASR) systems. By adopting romanization encoding alongside a balanced concatenated tokenizer within a FastConformer-RNNT framework equipped with a Roman2Char module, we significantly reduce vocabulary and output dimensions, enabling larger training batches and reduced memory consumption. Our method decouples acoustic modeling and language modeling, enhancing the flexibility and adaptability of the system. In our study, applying this method to Mandarin-English ASR resulted in a remarkable 63.51% vocabulary reduction and notable performance gains of 13.72% and 15.03% on SEAME code-switching benchmarks. Ablation studies on Mandarin-Korean and Mandarin-Japanese highlight our method's strong capability to address the complexities of other script-heavy languages, paving the way for more versatile and effective multilingual ASR systems.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 40 references that are in the map. First posted on arXiv 5 Jul 2024.

Builds on

30 of 40 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  4. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  5. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  6. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 364
  7. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  8. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 125
  9. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 120
  10. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 56
  11. 2019Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentencescited by 55
  12. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
18 more
  1. 2023Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalizationcited by 43
  2. 2018On the End-to-End Solution to Mandarin-English Code-switching Speech Recognitioncited by 30
  3. 2020Multi-Encoder-Decoder Transformer for Code-Switching Speech Recognitioncited by 28
  4. 2022Reducing Language confusion for Code-switching Speech Recognition with Token-level Language Diarizationcited by 28
  5. 2020Transformer-Transducers for Code-Switched Speech Recognitioncited by 27
  6. 2023Romanization-based Large-scale Adaptation of Multilingual Language Modelscited by 21
  7. 2019Is Word Segmentation Necessary for Deep Learning of Chinese Representations?cited by 18
  8. 2022Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilitiescited by 16
  9. 2019End-to-End Code-Switching ASR for Low-Resourced Language Pairscited by 15
  10. 2018A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinesecited by 10
  11. 2021Improving Code-switching Language Modeling with Artificially Generated Texts using Cycle-consistent Adversarial Networkscited by 9
  12. 2021Non-autoregressive Mandarin-English Code-switching Speech Recognitioncited by 8
  13. 2023Unified model for code-switching speech recognition and language identification based on a concatenated tokenizercited by 8
  14. 2021Sub-Character Tokenization for Chinese Pretrained Language Modelscited by 7
  15. 2021Decoupling recognition and transcription in Mandarin ASRcited by 6
  16. 2019Exploring Lexicon-Free Modeling Units for End-to-End Korean and Korean-English Code-Switching Speech Recognitioncited by 5
  17. 2022Bilingual End-to-End ASR with Byte-Level Subwordscited by 4
  18. 2022On the Effectiveness of Pinyin-Character Dual-Decoding for End-to-End Mandarin Chinese ASRcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Enhancing Cross-Lingual Transfer through Reversible Transliteration: A Huffman-Based Approach for Low-Resource Languagescited by 1