mlsanityData to 7 Oct 2026

FLToP CTC: Frame-Level Token Pruning via Relative Threshold for Efficient and Memory-Saving Decoding on Diverse Platforms

arXiv 2510.09085 ↗First posted 10 Oct 2025cs.LG cs.SD eess.AScited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

CTC-based ASR systems face computational and memory bottlenecks in resource-limited environments. Traditional CTC decoders, requiring up to 90% of processing time in systems (e.g., wav2vec2-large on L4 GPUs), face inefficiencies due to exhaustive token-level operations. This paper introduces Frame Level Token Pruning for Connectionist Temporal Classification (FLToP CTC), a novel decoding algorithm that employs frame-level token pruning guided by a relative threshold probability. By dynamically eliminating low-probability tokens per frame, FLToP CTC reduces compute and memory demands while maintaining negligible WER degradation. On LibriSpeech, FLToP CTC achieves a 10.5x runtime speedup and 2.78x memory reduction versus standard CTC decoders. Its simplicity enables seamless integration into CTC decoders across platforms (CPUs, GPUs, etc.). FLToP CTC addresses CTC bottlenecks, offering scalability for resource-limited environments and realtime applications, enhancing speech recognition accessibility and efficiency.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 13 of its 23 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

13 of 23 references.

References · 13

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  5. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  6. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  7. 2022Flashlight: Enabling Innovation in Tools for Machine Learningcited by 25
  8. 2018C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAscited by 21
  9. 2022Accelerating RNN-T Training and Inference Using CTC guidancecited by 17
  10. 2024Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPUcited by 12
  11. 2021FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularizationcited by 11
  12. 2023GPU-Accelerated WFST Beam Search Decoder for CTC-based Speech Recognitioncited by 4
1 more
  1. 2019A Hardware-Oriented and Memory-Efficient Method for CTC Decodingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Blank Collapse: Compressing CTC emission for the faster decoding
  2. 2026Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding
  3. 2026Elastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding
  4. 2025Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding
  5. 2026SwiftKV: An Edge-Oriented Attention Algorithm and Multi-Head Accelerator for Fast, Efficient LLM Decoding
  6. 2025AsyncSpade: Efficient Test-Time Scaling with Asynchronous Sparse Decoding
  7. 2026SinkRouter: Sink-Aware Routing for Efficient Long-Context Decoding in Large Language and Multimodal Models
  8. 2025CoDec: Prefix-Shared Decoding Kernel for LLMs