mlsanityData to 14 Oct 2026

Text-Utilization for Encoder-dominated Speech Recognition Models

arXiv 2604.26514 ↗First posted 29 Apr 2026cs.CL cs.AI cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper investigates efficient methods for utilizing text-only data to improve speech recognition, focusing on encoder-dominated models that facilitate faster recognition. We provide a comprehensive comparison of techniques to integrate text-only data, including modality matching and dynamic downsampling to reach text-level representations within the encoder. Our experiments on the LibriSpeech corpus show that a larger encoder with a smaller decoder can equal or surpass the performance of architectures with larger decoders. We demonstrate that simple configurations, such as random duration models, are often more effective than complex alternatives, significantly simplifying the training pipeline. All code and recipes are made publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 23 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

19 of 23 references.

References · 19

  1. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 180
  2. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 150
  3. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  4. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  5. 2022MAESTRO: Matched Speech Text Representations through Modality Matchingcited by 79
  6. 2021SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Trainingcited by 69
  7. 2022Unified Speech-Text Pre-training for Speech Translation and Recognitioncited by 64
  8. 2021Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictionscited by 61
  9. 2019Generating Synthetic Audio Data for Attention-Based Speech Recognition Systemscited by 47
  10. 2021CTC-based Compression for Direct Speech Translationcited by 45
  11. 2022SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-trainingcited by 40
  12. 2022JOIST: A Joint Speech and Text Streaming Model For ASRcited by 24
7 more
  1. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
  2. 2023Understanding Shared Speech-Text Representationscited by 6
  3. 2024Paraformer-v2: An improved non-autoregressive transformer for noise-robust speech recognitioncited by 5
  4. 2024CJST: CTC Compressor based Joint Speech and Text Training for Decoder-Only ASRcited by 3
  5. 2024ASTRA: Aligning Speech and Text Representations for Asr without Samplingcited by 2
  6. 2025A Non-autoregressive Model for Joint STT and TTScited by 1
  7. 2025Reproducing and Dissecting Denoising Language Models for Speech Recognitioncited by 1

Built on by

Not cited yet.