# Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Network

arXiv: https://arxiv.org/abs/2104.11127 · HTML: https://mlsanity.com/paper/2104.11127/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation](https://mlsanity.com/paper/1805.04699/) (2018)
- [SLURP: A Spoken Language Understanding Resource Package](https://mlsanity.com/paper/2011.13205/) (2020)
- [An analysis of incorporating an external language model into a sequence-to-sequence model](https://mlsanity.com/paper/1712.01996/) (2017)
- [Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer](https://mlsanity.com/paper/1801.00841/) (2018)
- [Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition](https://mlsanity.com/paper/2011.01991/) (2020)
- [Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capability](https://mlsanity.com/paper/2007.15188/) (2020)
- [Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces](https://mlsanity.com/paper/2005.09150/) (2020)

## Built on by (most cited first, 19 of 19)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Factorized Neural Transducer for Efficient Language Model Adaptation](https://mlsanity.com/paper/2110.01500/) (2021)
- [Integrating Text Inputs For Training and Adapting RNN Transducer ASR Models](https://mlsanity.com/paper/2202.13155/) (2022)
- [Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition](https://mlsanity.com/paper/2110.05354/) (2021)
- [On Language Model Integration for RNN Transducer based Speech Recognition](https://mlsanity.com/paper/2110.06841/) (2021)
- [Modular Hybrid Autoregressive Transducer](https://mlsanity.com/paper/2210.17049/) (2022)
- [CB-Conformer: Contextual biasing Conformer for biased word recognition](https://mlsanity.com/paper/2304.09607/) (2023)
- [JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition](https://mlsanity.com/paper/2302.08583/) (2023)
- [A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Data](https://mlsanity.com/paper/2206.13240/) (2022)
- [Adaptive Discounting of Implicit Language Models in RNN-Transducers](https://mlsanity.com/paper/2203.02317/) (2022)
- [Retraining-free Customized ASR for Enharmonic Words Based on a Named-Entity-Aware Model and Phoneme Similarity Estimation](https://mlsanity.com/paper/2305.17846/) (2023)
- [Text-only Domain Adaptation using Unified Speech-Text Representation in Transducer](https://mlsanity.com/paper/2306.04076/) (2023)
- [Corpus Synthesis for Zero-shot ASR domain Adaptation using Large Language Models](https://mlsanity.com/paper/2309.10707/) (2023)
- [Improving RNN-Transducers with Acoustic LookAhead](https://mlsanity.com/paper/2307.05006/) (2023)
- [Effective Text Adaptation for LLM-based ASR through Soft Prompt Fine-Tuning](https://mlsanity.com/paper/2412.06967/) (2024)
- [RASR2: The RWTH ASR Toolkit for Generic Sequence-to-sequence Speech Recognition](https://mlsanity.com/paper/2305.17782/) (2023)
- [Text Injection for Neural Contextual Biasing](https://mlsanity.com/paper/2406.02921/) (2024)
- [Improved Factorized Neural Transducer Model For text-only Domain Adaptation](https://mlsanity.com/paper/2309.09524/) (2023)
- [Multiple-hypothesis RNN-T Loss for Unsupervised Fine-tuning and Self-training of Neural Transducer](https://mlsanity.com/paper/2207.14736/) (2022)

## Abstract (arXiv)

Adaption of end-to-end speech recognition systems to new tasks is known to be challenging. A number of solutions have been proposed which apply external language models with various fusion methods, possibly with a combination of two-pass decoding. Also TTS systems have been used to generate adaptation data for the end-to-end models. In this paper we show that RNN-transducer models can be effectively adapted to new domains using only small amounts of textual data. By taking advantage of model's inherent structure, where the prediction network is interpreted as a language model, we can apply fast adaptation to the model. Adapting the model avoids the need for complicated decoding time fusions and external language models. Using appropriate regularization, the prediction network can be adapted to new domains while still retaining good generalization capabilities. We show with multiple ASR evaluation tasks how this method can provide relative gains of 10-45% in target task WER. We also share insights how RNN-transducer prediction network performs as a language model.