# Bifocal Neural ASR: Exploiting Keyword Spotting for Inference Optimization

arXiv: https://arxiv.org/abs/2108.01704 · HTML: https://mlsanity.com/paper/2108.01704/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2021.

## Built on by (most cited first, 7 of 7)

- [I3D: Transformer architectures with input-dependent dynamic depth for speech recognition](https://mlsanity.com/paper/2303.07624/) (2023)
- [Dual-Encoder Architecture with Encoder Selection for Joint Close-Talk and Far-Talk Speech Recognition](https://mlsanity.com/paper/2109.08744/) (2021)
- [Sub-8-bit quantization for on-device speech recognition: a regularization-free approach](https://mlsanity.com/paper/2210.09188/) (2022)
- [Compute Cost Amortized Transformer for Streaming ASR](https://mlsanity.com/paper/2207.02393/) (2022)
- [Dual-Attention Neural Transducers for Efficient Wake Word Spotting in Speech Recognition](https://mlsanity.com/paper/2304.01905/) (2023)
- [Latency Control for Keyword Spotting](https://mlsanity.com/paper/2206.07261/) (2022)
- [Accelerator-Aware Training for Transducer-Based Speech Recognition](https://mlsanity.com/paper/2305.07778/) (2023)

## Abstract (arXiv)

We present Bifocal RNN-T, a new variant of the Recurrent Neural Network Transducer (RNN-T) architecture designed for improved inference time latency on speech recognition tasks. The architecture enables a dynamic pivot for its runtime compute pathway, namely taking advantage of keyword spotting to select which component of the network to execute for a given audio frame. To accomplish this, we leverage a recurrent cell we call the Bifocal LSTM (BFLSTM), which we detail in the paper. The architecture is compatible with other optimization strategies such as quantization, sparsification, and applying time-reduction layers, making it especially applicable for deployed, real-time speech recognition settings. We present the architecture and report comparative experimental results on voice-assistant speech recognition tasks. Specifically, we show our proposed Bifocal RNN-T can improve inference cost by 29.1% with matching word error rates and only a minor increase in memory size.