mlsanityData to 7 Oct 2026

Text-only Domain Adaptation using Unified Speech-Text Representation in Transducer

arXiv 2306.04076 ↗First posted 7 Jun 2023cs.CL cs.SD eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

Domain adaptation using text-only corpus is challenging in end-to-end(E2E) speech recognition. Adaptation by synthesizing audio from text through TTS is resource-consuming. We present a method to learn Unified Speech-Text Representation in Conformer Transducer(USTR-CT) to enable fast domain adaptation using the text-only corpus. Different from the previous textogram method, an extra text encoder is introduced in our work to learn text representation and is removed during inference, so there is no modification for online deployment. To improve the efficiency of adaptation, single-step and multi-step adaptations are also explored. The experiments on adapting LibriSpeech to SPGISpeech show the proposed method reduces the word error rate(WER) by relatively 44% on the target domain, which is better than those of TTS method and textogram method. Also, it is shown the proposed method can be combined with internal language model estimation(ILME) to further improve the performance.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

33 of 39 references.

References · 33

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  5. 2015Attention-Based Models for Speech Recognitioncited by 400
  6. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  7. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 210
  8. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  9. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  10. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  11. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  12. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
21 more
  1. 2020Hybrid Autoregressive Transducer (hat)cited by 106
  2. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 99
  3. 2017Cold Fusion: Training Seq2Seq Models Together with Language Modelscited by 87
  4. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  5. 2022MAESTRO: Matched Speech Text Representations through Modality Matchingcited by 79
  6. 2021SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognitioncited by 77
  7. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  8. 2021SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Trainingcited by 69
  9. 2022Unified Speech-Text Pre-training for Speech Translation and Recognitioncited by 64
  10. 2020Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capabilitycited by 63
  11. 2020On the Comparison of Popular End-to-End Models for Large Scale Speech Recognitioncited by 62
  12. 2020Cascaded encoders for unifying streaming and non-streaming ASRcited by 51
  13. 2021Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognitioncited by 37
  14. 2020Improving Tail Performance of a Deliberation E2E ASR Model Using a Large Text Corpuscited by 24
  15. 2022JOIST: A Joint Speech and Text Streaming Model For ASRcited by 24
  16. 2022Integrating Text Inputs For Training and Adapting RNN Transducer ASR Modelscited by 20
  17. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
  18. 2021Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Networkcited by 19
  19. 2021Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognitioncited by 17
  20. 2018Advancing Acoustic-to-Word CTC Modelcited by 15
  21. 2023Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generatorcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 2; 2026: 1.

5 papers.

  1. 2023Improving Large-scale Deep Biasing with Phoneme Features and Text-only Data in Streaming Transducercited by 6
  2. 2024An efficient text augmentation approach for contextualized Mandarin speech recognitioncited by 2
  3. 2025AsyncSwitch: Asynchronous Text-Speech Adaptation for Code-Switched ASRcited by 1
  4. 2025Enhanced Hybrid Transducer and Attention Encoder Decoder with Text Datacited by 1
  5. 2026PTC-Bias: Phoneme-Level Temporal Competition for Bias Retrieval and Post-Decoding Correction in Speech LLMscited by 0