# kNN-CTC: Enhancing ASR via Retrieval of CTC Pseudo Labels

arXiv: https://arxiv.org/abs/2312.13560 · HTML: https://mlsanity.com/paper/2312.13560/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2023.

## Built on by (most cited first, 8 of 8)

- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [Enhancing Dysarthric Speech Recognition for Unseen Speakers via Prototype-Based Adaptation](https://mlsanity.com/paper/2407.18461/) (2024)
- [LA-RAG:Enhancing LLM-based ASR Accuracy with Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.08597/) (2024)
- [Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Review](https://mlsanity.com/paper/2507.07741/) (2025)
- [Optimizing Contextual Speech Recognition Using Vector Quantization for Efficient Retrieval](https://mlsanity.com/paper/2411.00664/) (2024)
- [Improving Zero-Shot Chinese-English Code-Switching ASR with kNN-CTC and Gated Monolingual Datastores](https://mlsanity.com/paper/2406.03814/) (2024)
- [MSDA: Combining Pseudo-labeling and Self-Supervision for Unsupervised Domain Adaptation in ASR](https://mlsanity.com/paper/2505.24656/) (2025)
- [Breaking Through the Spike: Spike Window Decoding for Accelerated and Precise Automatic Speech Recognition](https://mlsanity.com/paper/2501.03257/) (2025)

## Abstract (arXiv)

The success of retrieval-augmented language models in various natural language processing (NLP) tasks has been constrained in automatic speech recognition (ASR) applications due to challenges in constructing fine-grained audio-text datastores. This paper presents kNN-CTC, a novel approach that overcomes these challenges by leveraging Connectionist Temporal Classification (CTC) pseudo labels to establish frame-level audio-text key-value pairs, circumventing the need for precise ground truth alignments. We further introduce a skip-blank strategy, which strategically ignores CTC blank frames, to reduce datastore size. kNN-CTC incorporates a k-nearest neighbors retrieval mechanism into pre-trained CTC ASR systems, achieving significant improvements in performance. By incorporating a k-nearest neighbors retrieval mechanism into pre-trained CTC ASR systems and leveraging a fine-grained, pruned datastore, kNN-CTC consistently achieves substantial improvements in performance under various experimental settings. Our code is available at https://github.com/NKU-HLT/KNN-CTC.