# WIND: Accelerated RNN-T Decoding with Windowed Inference for Non-blank Detection

arXiv: https://arxiv.org/abs/2505.13765 · HTML: https://mlsanity.com/paper/2505.13765/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 14 of 14)

- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://mlsanity.com/paper/2305.05084/) (2023)
- [Efficient Sequence Transduction by Jointly Predicting Tokens and Durations](https://mlsanity.com/paper/2304.06795/) (2023)
- [Accelerating RNN Transducer Inference via One-Step Constrained Beam Search](https://mlsanity.com/paper/2002.03577/) (2020)
- [Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPU](https://mlsanity.com/paper/2406.03791/) (2024)
- [Multi-blank Transducers for Speech Recognition](https://mlsanity.com/paper/2211.03541/) (2022)
- [Label-Looping: Highly Efficient Decoding for Transducers](https://mlsanity.com/paper/2406.06220/) (2024)
- [Blank-regularized CTC for Frame Skipping in Neural Transducer](https://mlsanity.com/paper/2305.11558/) (2023)
- [A Token-Wise Beam Search Algorithm for RNN-T](https://mlsanity.com/paper/2302.14357/) (2023)

## Built on by (most cited first, 1 of 1)

- [Chunk-wise Attention Transducers for Fast and Accurate Streaming Speech-to-Text](https://mlsanity.com/paper/2602.24245/) (2026)

## Abstract (arXiv)

We propose Windowed Inference for Non-blank Detection (WIND), a novel strategy that significantly accelerates RNN-T inference without compromising model accuracy. During model inference, instead of processing frames sequentially, WIND processes multiple frames simultaneously within a window in parallel, allowing the model to quickly locate non-blank predictions during decoding, resulting in significant speed-ups. We implement WIND for greedy decoding, batched greedy decoding with label-looping techniques, and also propose a novel beam-search decoding method. Experiments on multiple datasets with different conditions show that our method, when operating in greedy modes, speeds up as much as 2.4X compared to the baseline sequential approach while maintaining identical Word Error Rate (WER) performance. Our beam-search algorithm achieves slightly better accuracy than alternative methods, with significantly improved speed. We will open-source our WIND implementation.