# Streaming Speech-to-Confusion Network Speech Recognition

arXiv: https://arxiv.org/abs/2306.03778 · HTML: https://mlsanity.com/paper/2306.03778/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 25 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 20 of 20)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020)
- [Contextual RNN-T For Open Domain ASR](https://mlsanity.com/paper/2006.03411/) (2020)
- [Streaming automatic speech recognition with the transformer model](https://mlsanity.com/paper/2001.02674/) (2020)
- [FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization](https://mlsanity.com/paper/2010.11148/) (2020)
- [Contextual Adapters for Personalized Speech Recognition in Neural Transducers](https://mlsanity.com/paper/2205.13660/) (2022)
- [Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping](https://mlsanity.com/paper/1902.06450/) (2019)
- [Improved Mask-CTC for Non-Autoregressive End-to-End ASR](https://mlsanity.com/paper/2010.13270/) (2020)
- [Transformer-based Online CTC/attention End-to-End Speech Recognition Architecture](https://mlsanity.com/paper/2001.08290/) (2020)
- [Less Is More: Improved RNN-T Decoding Using Limited Label Context and Path Merging](https://mlsanity.com/paper/2012.06749/) (2020)
- [Synchronous Transformers for End-to-End Speech Recognition](https://mlsanity.com/paper/1912.02958/) (2019)
- [Low Latency End-to-End Streaming Speech Recognition with a Scout Network](https://mlsanity.com/paper/2003.10369/) (2020)
- [Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models](https://mlsanity.com/paper/2107.09428/) (2021)
- [Unified End-to-End Speech Recognition and Endpointing for Fast and Efficient Speech Systems](https://mlsanity.com/paper/2211.00786/) (2022)
- [Towards Online End-to-end Transformer Automatic Speech Recognition](https://mlsanity.com/paper/1910.11871/) (2019)
- [VQ-T: RNN Transducers using Vector-Quantized Prediction Network States](https://mlsanity.com/paper/2208.01818/) (2022)

## Abstract (arXiv)

In interactive automatic speech recognition (ASR) systems, low-latency requirements limit the amount of search space that can be explored during decoding, particularly in end-to-end neural ASR. In this paper, we present a novel streaming ASR architecture that outputs a confusion network while maintaining limited latency, as needed for interactive applications. We show that 1-best results of our model are on par with a comparable RNN-T system, while the richer hypothesis set allows second-pass rescoring to achieve 10-20% lower word error rate on the LibriSpeech task. We also show that our model outperforms a strong RNN-T baseline on a far-field voice assistant task.