mlsanityData to 7 Oct 2026

Modular Hybrid Autoregressive Transducer

arXiv 2210.17049 ↗First posted 31 Oct 2022cs.CL cs.AI cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Text-only adaptation of a transducer model remains challenging for end-to-end speech recognition since the transducer has no clearly separated acoustic model (AM), language model (LM) or blank model. In this work, we propose a modular hybrid autoregressive transducer (MHAT) that has structurally separated label and blank decoders to predict label and blank distributions, respectively, along with a shared acoustic encoder. The encoder and label decoder outputs are directly projected to AM and internal LM scores and then added to compute label posteriors. We train MHAT with an internal LM loss and a HAT loss to ensure that its internal LM becomes a standalone neural LM that can be effectively adapted to text. Moreover, text adaptation of MHAT fosters a much better LM fusion than internal LM subtraction-based methods. On Google's large-scale production data, a multi-domain MHAT adapted with 100B sentences achieves relative WER reductions of up to 12.4% without LM fusion and 21.5% with LM fusion from 400K-hour trained HAT.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 32 of its 61 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

32 of 61 references.

References · 32

  1. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  2. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  4. 2015Listen, Attend and Spellcited by 542
  5. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  6. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
  7. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  8. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  9. 2020Hybrid Autoregressive Transducer (hat)cited by 106
  10. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 99
  11. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  12. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
20 more
  1. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  2. 2020Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capabilitycited by 63
  3. 2019Two-Pass End-to-End Speech Recognitioncited by 62
  4. 2020Cascaded encoders for unifying streaming and non-streaming ASRcited by 51
  5. 2021Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognitioncited by 37
  6. 2021Tied & Reduced RNN-T Decodercited by 35
  7. 2020A New Training Pipeline for an Improved Neural Transducercited by 34
  8. 2021Factorized Neural Transducer for Efficient Language Model Adaptationcited by 31
  9. 2021Librispeech Transducer Model with Internal Language Model Prior Correctioncited by 26
  10. 2018Speaker-Invariant Training via Adversarial Learningcited by 23
  11. 2020Domain Adaptation via Teacher-Student Learning for End-to-End Speech Recognitioncited by 21
  12. 2019RNN-T For Latency Controlled ASR With Improved Beam Searchcited by 19
  13. 2021Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Networkcited by 19
  14. 2019Improving Performance of End-to-End ASR on Numeric Sequencescited by 17
  15. 2021Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognitioncited by 17
  16. 2016Invariant Representations for Noisy Speech Recognitioncited by 12
  17. 2017Unsupervised Adaptation with Domain Separation Networks for Robust Speech Recognitioncited by 10
  18. 2019Adversarial Speaker Adaptationcited by 10
  19. 2019Speaker Adaptation for Attention-Based End-to-End Speech Recognitioncited by 10
  20. 2020L-Vector: Neural Label Embedding for Domain Adaptationcited by 9

Built on by

Cited by year

Citing papers per year: 2023: 9; 2024: 4; 2025: 4.

17 papers.

Sort:

  1. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  2. 2023Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generatorcited by 11
  3. 2023JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognitioncited by 9
  4. 2025NGPU-LM: GPU-Accelerated N-Gram Language Model for Context-Biasing in Greedy ASR Decodingcited by 7
  5. 2023Understanding Shared Speech-Text Representationscited by 6
  6. 2024Effective Text Adaptation for LLM-based ASR through Soft Prompt Fine-Tuningcited by 4
  7. 2024Transducer-Llama: Integrating LLMs into Streamable Transducer-based Speech Recognitioncited by 4
  8. 2025Pushing the Limits of Beam Search Decoding for Transducer-based ASR modelscited by 4
  9. 2023Decoupled Structure for Improved Adaptability of End-to-End Modelscited by 3
  10. 2024Effective internal language model training and fusion for factorized transducer modelcited by 3
  11. 2024Text Injection for Neural Contextual Biasingcited by 3
  12. 2023External Language Model Integration for Factorized Neural Transducerscited by 2
5 more
  1. 2023Improved Factorized Neural Transducer Model For text-only Domain Adaptationcited by 2
  2. 2023Text Injection for Capitalization and Turn-Taking Prediction in Speech Modelscited by 1
  3. 2023Hybrid Attention-based Encoder-decoder Model for Efficient Language Model Adaptationcited by 1
  4. 2025Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducerscited by 1
  5. 2025RNN-Transducer-based Losses for Speech Recognition on Noisy Targetscited by 1