# Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text

arXiv: https://arxiv.org/abs/2602.24245 · HTML: https://mlsanity.com/paper/2602.24245/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 27 Feb 2026.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://mlsanity.com/paper/2305.05084/) (2023)
- [Efficient Sequence Transduction by Jointly Predicting Tokens and Durations](https://mlsanity.com/paper/2304.06795/) (2023)
- [Large-Scale Streaming End-to-End Speech Translation with Neural Transducers](https://mlsanity.com/paper/2204.05352/) (2022)
- [Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2312.17279/) (2023)
- [Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPU](https://mlsanity.com/paper/2406.03791/) (2024)
- [Multi-blank Transducers for Speech Recognition](https://mlsanity.com/paper/2211.03541/) (2022)
- [Label-Looping: Highly Efficient Decoding for Transducers](https://mlsanity.com/paper/2406.06220/) (2024)
- [Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognition](https://mlsanity.com/paper/2309.08436/) (2023)
- [HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR](https://mlsanity.com/paper/2410.02597/) (2024)
- [WIND: Accelerated RNN-T Decoding with Windowed Inference for Non-blank Detection](https://mlsanity.com/paper/2505.13765/) (2025)

## Abstract (arXiv)

We propose Chunk-wise Attention Transducer (CHAT), a novel extension to RNN-T models that processes audio in fixed-size chunks while employing cross-attention within each chunk. This hybrid approach maintains RNN-T's streaming capability while introducing controlled flexibility for local alignment modeling. CHAT significantly reduces the temporal dimension that RNN-T must handle, yielding substantial efficiency improvements: up to 46.2% reduction in peak training memory, up to 1.36X faster training, and up to 1.69X faster inference. Alongside these efficiency gains, CHAT achieves consistent accuracy improvements over RNN-T across multiple languages and tasks -- up to 6.3% relative WER reduction for speech recognition and up to 18.0% BLEU improvement for speech translation. The method proves particularly effective for speech translation, where RNN-T's strict monotonic alignment hurts performance. Our results demonstrate that the CHAT model offers a practical solution for deploying more capable streaming speech models without sacrificing real-time constraints.