# Spiralformer: Low Latency Encoder for Streaming Speech Recognition with Circular Layer Skipping and Early Exiting

arXiv: https://arxiv.org/abs/2510.00982 · HTML: https://mlsanity.com/paper/2510.00982/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Audio and omni-modal token compression, one of 12 directions in Efficient LLM inference and serving.
- Builds on 27 of its 33 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Placed in

- Efficient LLM inference and serving / [Audio and omni-modal token compression](https://mlsanity.com/inference/audio-and-omni-modal-token-compression/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020)
- [On the Effect of Dropping Layers of Pre-trained Transformer Models](https://mlsanity.com/paper/2004.03844/) (2020)
- [Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition](https://mlsanity.com/paper/2010.10759/) (2020)
- [Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer](https://mlsanity.com/paper/1801.00841/) (2018)
- [E-Branchformer: Branchformer with Enhanced merging for speech recognition](https://mlsanity.com/paper/2210.00077/) (2022)
- [EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding](https://mlsanity.com/paper/1507.08240/) (2015)
- [Streaming automatic speech recognition with the transformer model](https://mlsanity.com/paper/2001.02674/) (2020)
- [FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization](https://mlsanity.com/paper/2010.11148/) (2020)
- [Towards Fast and Accurate Streaming End-to-End ASR](https://mlsanity.com/paper/2004.11544/) (2020)
- [Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping](https://mlsanity.com/paper/1902.06450/) (2019)
- [Transformer ASR with Contextual Block Processing](https://mlsanity.com/paper/1910.07204/) (2019)
- [Transformer-based Online CTC/attention End-to-End Speech Recognition Architecture](https://mlsanity.com/paper/2001.08290/) (2020)
- [On the Usefulness of Self-Attention for Automatic Speech Recognition with Transformers](https://mlsanity.com/paper/2011.04906/) (2020)
- [Dissecting User-Perceived Latency of On-Device E2E Speech Recognition](https://mlsanity.com/paper/2104.02207/) (2021)
- [Layer Pruning on Demand with Intermediate CTC](https://mlsanity.com/paper/2106.09216/) (2021)
- [HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition](https://mlsanity.com/paper/2204.06328/) (2022)
- [Decoder-only Architecture for Streaming End-to-end Speech Recognition](https://mlsanity.com/paper/2406.16107/) (2024)
- [TrimTail: Low-Latency Streaming ASR with Simple but Effective Spectrogram-Level Length Penalty](https://mlsanity.com/paper/2211.00522/) (2022)
- [Avoid Overthinking in Self-Supervised Models for Speech Recognition](https://mlsanity.com/paper/2211.08989/) (2022)
- [Semi-Autoregressive Streaming ASR With Label Context](https://mlsanity.com/paper/2309.10926/) (2023)
- [Transformer-based Streaming ASR with Cumulative Attention](https://mlsanity.com/paper/2203.05736/) (2022)

## Abstract (arXiv)

For streaming speech recognition, a Transformer-based encoder has been widely used with block processing. Although many studies addressed improving emission latency of transducers, little work has been explored for improving encoding latency of the block processing. We seek to reduce latency by frequently emitting a chunk with a small shift rather than scarce large-chunk emissions, resulting in higher computational costs. To efficiently compute with the small chunk shift, we propose a new encoder, Spiralformer, tailored for block processing by combining layer dropping and early exiting. We skip layer computation in a cyclic manner and shift the computed layer in each block spirally, which completes computation for all the layers over the block processing. Experimentally, we observed that our method achieved 21.6% reduction in the averaged token emission delay in Librispeech, and 7.0% in CSJ, compared with the baseline with similar computational cost and word error rates.