# Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognition

arXiv: https://arxiv.org/abs/2309.08436 · HTML: https://mlsanity.com/paper/2309.08436/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 52 references that are in the map.
- First posted on arXiv 15 Sep 2023.

## Builds on (most cited first, 30 of 40)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Correcting Length Bias in Neural Machine Translation](https://mlsanity.com/paper/1808.10006/) (2018)
- [Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition](https://mlsanity.com/paper/2010.10759/) (2020)
- [Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset](https://mlsanity.com/paper/2010.11395/) (2020)
- [Monotonic Chunkwise Attention](https://mlsanity.com/paper/1712.05382/) (2017)
- [Recognizing long-form speech using streaming end-to-end models](https://mlsanity.com/paper/1910.11455/) (2019)
- [Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations](https://mlsanity.com/paper/1606.01305/) (2016)
- [Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM](https://mlsanity.com/paper/1706.02737/) (2017)
- [A comparison of end-to-end models for long-form speech recognition](https://mlsanity.com/paper/1911.02242/) (2019)
- [Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition](https://mlsanity.com/paper/2012.05481/) (2020)
- [Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard](https://mlsanity.com/paper/2001.07263/) (2020)
- [A New Training Pipeline for an Improved Neural Transducer](https://mlsanity.com/paper/2005.09319/) (2020)
- [Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping](https://mlsanity.com/paper/1902.06450/) (2019)
- [RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition](https://mlsanity.com/paper/1805.05225/) (2018)
- [Transformer ASR with Contextual Block Processing](https://mlsanity.com/paper/1910.07204/) (2019)
- [Librispeech Transducer Model with Internal Language Model Prior Correction](https://mlsanity.com/paper/2104.03006/) (2021)
- [Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Models](https://mlsanity.com/paper/2104.05544/) (2021)
- [Synchronous Transformers for End-to-End Speech Recognition](https://mlsanity.com/paper/1912.02958/) (2019)
- [Low Latency End-to-End Streaming Speech Recognition with a Scout Network](https://mlsanity.com/paper/2003.10369/) (2020)
- [Length bias in Encoder Decoder Models and a Case for Global Conditioning](https://mlsanity.com/paper/1606.03402/) (2016)
- [Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognition](https://mlsanity.com/paper/2107.01269/) (2021)

## Built on by (most cited first, 7 of 7)

- [Efficient Streaming LLM for Speech Recognition](https://mlsanity.com/paper/2410.03752/) (2024)
- [The Conformer Encoder May Reverse the Time Dimension](https://mlsanity.com/paper/2410.00680/) (2024)
- [Pace: Physics-Aware Attentive Temporal Convolutional Network for Battery Health Estimation](https://mlsanity.com/paper/2512.11332/) (2025)
- [Weak Alignment Supervision from Hybrid Model Improves End-to-end ASR](https://mlsanity.com/paper/2311.14835/) (2023)
- [Alternating Weak Triphone/BPE Alignment Supervision from Hybrid Model Improves End-to-End ASR](https://mlsanity.com/paper/2402.15594/) (2024)
- [Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text](https://mlsanity.com/paper/2602.24245/) (2026)
- [SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2603.10005/) (2026)

## Abstract (arXiv)

We study a streamable attention-based encoder-decoder model in which either the decoder, or both the encoder and decoder, operate on pre-defined, fixed-size windows called chunks. A special end-of-chunk (EOC) symbol advances from one chunk to the next chunk, effectively replacing the conventional end-of-sequence symbol. This modification, while minor, situates our model as equivalent to a transducer model that operates on chunks instead of frames, where EOC corresponds to the blank symbol. We further explore the remaining differences between a standard transducer and our model. Additionally, we examine relevant aspects such as long-form speech generalization, beam size, and length normalization. Through experiments on Librispeech and TED-LIUM-v2, and by concatenating consecutive sequences for long-form trials, we find that our streamable model maintains competitive performance compared to the non-streamable variant and generalizes very well to long-form speech.