# Text-Utilization for Encoder-dominated Speech Recognition Models

arXiv: https://arxiv.org/abs/2604.26514 · HTML: https://mlsanity.com/paper/2604.26514/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 23 references that are in the map.
- First posted on arXiv 29 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing](https://mlsanity.com/paper/2110.07205/) (2021)
- [What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?](https://mlsanity.com/paper/2204.05832/) (2022)
- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [mSLAM: Massively multilingual joint pre-training for speech and text](https://mlsanity.com/paper/2202.01374/) (2022)
- [MAESTRO: Matched Speech Text Representations through Modality Matching](https://mlsanity.com/paper/2204.03409/) (2022)
- [SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training](https://mlsanity.com/paper/2110.10329/) (2021)
- [Unified Speech-Text Pre-training for Speech Translation and Recognition](https://mlsanity.com/paper/2204.05409/) (2022)
- [Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictions](https://mlsanity.com/paper/2104.02724/) (2021)
- [Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems](https://mlsanity.com/paper/1912.09257/) (2019)
- [CTC-based Compression for Direct Speech Translation](https://mlsanity.com/paper/2102.01578/) (2021)
- [SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training](https://mlsanity.com/paper/2210.03730/) (2022)
- [JOIST: A Joint Speech and Text Streaming Model For ASR](https://mlsanity.com/paper/2210.07353/) (2022)
- [Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systems](https://mlsanity.com/paper/2203.00006/) (2022)
- [Understanding Shared Speech-Text Representations](https://mlsanity.com/paper/2304.14514/) (2023)
- [Paraformer-v2: An improved non-autoregressive transformer for noise-robust speech recognition](https://mlsanity.com/paper/2409.17746/) (2024)
- [CJST: CTC Compressor based Joint Speech and Text Training for Decoder-Only ASR](https://mlsanity.com/paper/2411.07607/) (2024)
- [ASTRA: Aligning Speech and Text Representations for Asr without Sampling](https://mlsanity.com/paper/2406.06664/) (2024)
- [A Non-autoregressive Model for Joint STT and TTS](https://mlsanity.com/paper/2501.09104/) (2025)
- [Reproducing and Dissecting Denoising Language Models for Speech Recognition](https://mlsanity.com/paper/2512.13576/) (2025)

## Abstract (arXiv)

This paper investigates efficient methods for utilizing text-only data to improve speech recognition, focusing on encoder-dominated models that facilitate faster recognition. We provide a comprehensive comparison of techniques to integrate text-only data, including modality matching and dynamic downsampling to reach text-level representations within the encoder. Our experiments on the LibriSpeech corpus show that a larger encoder with a smaller decoder can equal or surpass the performance of architectures with larger decoders. We demonstrate that simple configurations, such as random duration models, are often more effective than complex alternatives, significantly simplifying the training pipeline. All code and recipes are made publicly available.