# HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR

arXiv: https://arxiv.org/abs/2410.02597 · HTML: https://mlsanity.com/paper/2410.02597/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 45 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://mlsanity.com/paper/2305.05084/) (2023)
- [The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage](https://mlsanity.com/paper/2111.09344/) (2021)
- [Zipformer: A faster and better encoder for automatic speech recognition](https://mlsanity.com/paper/2310.11230/) (2023)
- [Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predict](https://mlsanity.com/paper/2005.08700/) (2020)
- [Transformer-Transducer: End-to-End Speech Recognition with Self-Attention](https://mlsanity.com/paper/1910.12977/) (2019)
- [Efficient Sequence Transduction by Jointly Predicting Tokens and Durations](https://mlsanity.com/paper/2304.06795/) (2023)
- [Deliberation Model Based Two-Pass End-to-End Speech Recognition](https://mlsanity.com/paper/2003.07962/) (2020)
- [Why does CTC result in peaky behavior?](https://mlsanity.com/paper/2105.14849/) (2021)
- [Improved Mask-CTC for Non-Autoregressive End-to-End ASR](https://mlsanity.com/paper/2010.13270/) (2020)
- [RNN-T For Latency Controlled ASR With Improved Beam Search](https://mlsanity.com/paper/1911.01629/) (2019)
- [Deliberation of Streaming RNN-Transducer by Non-autoregressive Decoding](https://mlsanity.com/paper/2112.11442/) (2021)
- [Fast and parallel decoding for transducer](https://mlsanity.com/paper/2211.00484/) (2022)
- [Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPU](https://mlsanity.com/paper/2406.03791/) (2024)
- [Multi-blank Transducers for Speech Recognition](https://mlsanity.com/paper/2211.03541/) (2022)
- [Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoring](https://mlsanity.com/paper/2109.04411/) (2021)
- [CTC-based Non-autoregressive Speech Translation](https://mlsanity.com/paper/2305.17358/) (2023)
- [Label-Looping: Highly Efficient Decoding for Transducers](https://mlsanity.com/paper/2406.06220/) (2024)
- [Streaming parallel transducer beam search with fast-slow cascaded encoders](https://mlsanity.com/paper/2203.15773/) (2022)
- [Semi-Autoregressive Streaming ASR With Label Context](https://mlsanity.com/paper/2309.10926/) (2023)

## Built on by (most cited first, 2 of 2)

- [Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text](https://mlsanity.com/paper/2602.24245/) (2026)
- [Proactive for Uncertainty: Cause-Aware Error Diagnosis and Interactive Clarification for Spoken Dialogue Systems](https://mlsanity.com/paper/2605.25404/) (2026)

## Abstract (arXiv)

We present Hybrid-Autoregressive INference TrANsducers (HAINAN), a novel architecture for speech recognition that extends the Token-and-Duration Transducer (TDT) model. Trained with randomly masked predictor network outputs, HAINAN supports both autoregressive inference with all network components and non-autoregressive inference without the predictor. Additionally, we propose a novel semi-autoregressive inference paradigm that first generates an initial hypothesis using non-autoregressive inference, followed by refinement steps where each token prediction is regenerated using parallelized autoregression on the initial hypothesis. Experiments on multiple datasets across different languages demonstrate that HAINAN achieves efficiency parity with CTC in non-autoregressive mode and with TDT in autoregressive mode. In terms of accuracy, autoregressive HAINAN outperforms TDT and RNN-T, while non-autoregressive HAINAN significantly outperforms CTC. Semi-autoregressive inference further enhances the model's accuracy with minimal computational overhead, and even outperforms TDT results in some cases. These results highlight HAINAN's flexibility in balancing accuracy and speed, positioning it as a strong candidate for real-world speech recognition applications.