Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models
Not on the map. Why?
Abstract (arXiv)
This paper explores speculative speech recognition (SSR), where we empower conventional automatic speech recognition (ASR) with speculation capabilities, allowing the recognizer to run ahead of audio. We introduce a metric for measuring SSR performance and we propose a model which does SSR by combining a RNN-Transducer-based ASR system with an audio-prefixed language model (LM). The ASR system transcribes ongoing audio and feeds the resulting transcripts, along with an audio-dependent prefix, to the LM, which speculates likely completions for the transcriptions. We experiment with a variety of ASR datasets on which show the efficacy our method and the feasibility of SSR as a method of reducing ASR latency.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 5 Jul 2024.
Builds on
22 of 33 references.
References · 22
- 2017Attention Is All You Needcited by 59,932
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
- 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
- 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
- 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
- 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
- 2021SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Networkcited by 83
10 more
- 2020A Better and Faster End-to-End Model for Streaming ASRcited by 68
- 2020FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularizationcited by 66
- 2019Two-Pass End-to-End Speech Recognitioncited by 62
- 2020Towards Fast and Accurate Streaming End-to-End ASRcited by 47
- 2020Multitask Training with Text Data for End-to-End Speech Recognitioncited by 19
- 2022USTED: Improving ASR with a Unified Speech and Text Encoder-Decodercited by 7
- 2023Lookahead When It Matters: Adaptive Non-causal Transformers for Streaming Neural Transducerscited by 6
- 2023Personalized Predictive ASR for Latency Reduction in Voice Assistantscited by 6
- 2023On-the-fly Text Retrieval for End-to-End ASR Adaptationcited by 4
- 2019Statistical Testing on ASR Performance via Blockwise Bootstrapcited by 3
Built on by
Citing papers per year: 2024: 2; 2025: 0; 2026: 1.
3 papers.