# GPU-Accelerated WFST Beam Search Decoder for CTC-based Speech Recognition

arXiv: https://arxiv.org/abs/2311.04996 · HTML: https://mlsanity.com/paper/2311.04996/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 20 references that are in the map.
- First posted on arXiv 8 Nov 2023.

## Builds on (most cited first, 7 of 7)

- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Flashlight: Enabling Innovation in Tools for Machine Learning](https://mlsanity.com/paper/2201.12465/) (2022)
- [CTC Variations Through New WFST Topologies](https://mlsanity.com/paper/2110.03098/) (2021)
- [Fast and parallel decoding for transducer](https://mlsanity.com/paper/2211.00484/) (2022)
- [GPU-Accelerated Viterbi Exact Lattice Decoder for Batched Online and Offline Speech Recognition](https://mlsanity.com/paper/1910.10032/) (2019)
- [A GPU-based WFST Decoder with Exact Lattice Generation](https://mlsanity.com/paper/1804.03243/) (2018)
- [Decoding with Finite-State Transducers on GPUs](https://mlsanity.com/paper/1701.03038/) (2017)

## Built on by (most cited first, 4 of 4)

- [Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter](https://mlsanity.com/paper/2406.07096/) (2024)
- [FlexCTC: GPU-powered CTC Beam Decoding With Advanced Contextual Abilities](https://mlsanity.com/paper/2508.07315/) (2025)
- [Breaking Through the Spike: Spike Window Decoding for Accelerated and Precise Automatic Speech Recognition](https://mlsanity.com/paper/2501.03257/) (2025)
- [FLToP CTC: Frame-Level Token Pruning via Relative Threshold for Efficient and Memory-Saving Decoding on Diverse Platforms](https://mlsanity.com/paper/2510.09085/) (2025)

## Abstract (arXiv)

While Connectionist Temporal Classification (CTC) models deliver state-of-the-art accuracy in automated speech recognition (ASR) pipelines, their performance has been limited by CPU-based beam search decoding. We introduce a GPU-accelerated Weighted Finite State Transducer (WFST) beam search decoder compatible with current CTC models. It increases pipeline throughput and decreases latency, supports streaming inference, and also supports advanced features like utterance-specific word boosting via on-the-fly composition. We provide pre-built DLPack-based python bindings for ease of use with Python-based machine learning frameworks at https://github.com/nvidia-riva/riva-asrlib-decoder. We evaluated our decoder for offline and online scenarios, demonstrating that it is the fastest beam search decoder for CTC models. In the offline scenario it achieves up to 7 times more throughput than the current state-of-the-art CPU decoder and in the online streaming scenario, it achieves nearly 8 times lower latency, with same or better word error rate.