# FastInject: Injecting Unpaired Text Data into CTC-based ASR training

arXiv: https://arxiv.org/abs/2312.09100 · HTML: https://mlsanity.com/paper/2312.09100/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2023.

## Built on by (most cited first, 3 of 3)

- [Temporal Order Preserved Optimal Transport-based Cross-modal Knowledge Transfer Learning for ASR](https://mlsanity.com/paper/2409.02239/) (2024)
- [New Insights into Optimal Alignment of Acoustic and Linguistic Representations for Knowledge Transfer in ASR](https://mlsanity.com/paper/2509.05609/) (2025)
- [Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving](https://mlsanity.com/paper/2607.01733/) (2026)

## Abstract (arXiv)

Recently, connectionist temporal classification (CTC)-based end-to-end (E2E) automatic speech recognition (ASR) models have achieved impressive results, especially with the development of self-supervised learning. However, E2E ASR models trained on paired speech-text data often suffer from domain shifts from training to testing. To alleviate this issue, this paper proposes a flat-start joint training method, named FastInject, which efficiently injects multi-domain unpaired text data into CTC-based ASR training. To maintain training efficiency, text units are pre-upsampled, and their representations are fed into the CTC model along with speech features. To bridge the modality gap between speech and text, an attention-based modality matching mechanism (AM3) is proposed, which retains the E2E flat-start training. Experiments show that the proposed FastInject gave a 22% relative WER reduction (WERR) for intra-domain Librispeech-100h data and 20% relative WERR on out-of-domain test sets.