mlsanityData to 7 Oct 2026

Improving Code-Switching and Named Entity Recognition in ASR with Speech Editing based Data Augmentation

arXiv 2306.08588 ↗First posted 14 Jun 2023cs.CL cs.SD eess.AScited by 2

Not on the map. Why?

Abstract (arXiv)

Recently, end-to-end (E2E) automatic speech recognition (ASR) models have made great strides and exhibit excellent performance in general speech recognition. However, there remain several challenging scenarios that E2E models are not competent in, such as code-switching and named entity recognition (NER). Data augmentation is a common and effective practice for these two scenarios. However, the current data augmentation methods mainly rely on audio splicing and text-to-speech (TTS) models, which might result in discontinuous, unrealistic, and less diversified speech. To mitigate these potential issues, we propose a novel data augmentation method by applying the text-based speech editing model. The augmented speech from speech editing systems is more coherent and diversified, also more akin to real speech. The experimental results on code-switching and NER tasks show that our proposed method can significantly outperform the audio splicing and neural TTS based data augmentation systems.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

33 of 47 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  3. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  4. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  5. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  6. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  7. 2018Deep context: end-to-end contextual speech recognitioncited by 136
  8. 2021Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusioncited by 100
  9. 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
  10. 2022Contextual Adapters for Personalized Speech Recognition in Neural Transducerscited by 57
  11. 2020The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Resultscited by 43
  12. 2020Using Synthetic Audio to Improve The Recognition of Out-Of-Vocabulary Words in End-To-End ASR Systemscited by 41
21 more
  1. 2022A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editingcited by 40
  2. 2020End-to-end Named Entity Recognition from English Speechcited by 36
  3. 2020Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognitioncited by 34
  4. 2022VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Featurecited by 32
  5. 2021EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusioncited by 31
  6. 2018Code-switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentationcited by 26
  7. 2018End-to-end contextual speech recognition using class language models and a token passing decodercited by 26
  8. 2021Fast Contextual Adaptation with Neural Associative Memory for On-Device Personalized Speech Recognitioncited by 25
  9. 2022Streaming End-to-End Multilingual Speech Recognition with Joint Language Identificationcited by 24
  10. 2022LAE: Language-Aware Encoder for Monolingual and Multilingual ASRcited by 21
  11. 2020Data Augmentation for End-to-end Code-switching Speech Recognitioncited by 20
  12. 2021On Addressing Practical Challenges for RNN-Transducercited by 19
  13. 2021Joint Modeling of Code-Switched and Monolingual ASR via Conditional Factorizationcited by 17
  14. 2022SpeechPainter: Text-conditioned Speech Inpaintingcited by 17
  15. 2019End-to-End Code-Switching ASR for Low-Resourced Language Pairscited by 15
  16. 2020Improving Low Resource Code-switched ASR using Augmented Code-switched TTScited by 15
  17. 2018Acoustic and Textual Data Augmentation for Improved ASR of Code-Switching Speechcited by 12
  18. 2022Textual Data Augmentation for Arabic-English Code-Switching Speech Recognitioncited by 10
  19. 2022RetrieverTTS: Modeling Decomposed Factors for Text-Based Speech Insertioncited by 10
  20. 2021Improving Code-switching Language Modeling with Artificially Generated Texts using Cycle-consistent Adversarial Networkscited by 9
  21. 2021Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narrationcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2024Hard-Synth: Synthesizing Diverse Hard Samples for ASR using Zero-Shot TTS and LLMcited by 1
  2. 2023Incorporating Class-based Language Model for Named Entity Recognition in Factorized Neural Transducercited by 0