# Evaluating Automatic Speech Recognition in an Incremental Setting

arXiv: https://arxiv.org/abs/2302.12049 · HTML: https://mlsanity.com/paper/2302.12049/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 16 references that are in the map.
- First posted on arXiv 23 Feb 2023.

## Builds on (most cited first, 2 of 2)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)

## Built on by (most cited first, 1 of 1)

- [Post-decoder Biasing for End-to-End Speech Recognition of Multi-turn Medical Interview](https://mlsanity.com/paper/2403.00370/) (2024)

## Abstract (arXiv)

The increasing reliability of automatic speech recognition has proliferated its everyday use. However, for research purposes, it is often unclear which model one should choose for a task, particularly if there is a requirement for speed as well as accuracy. In this paper, we systematically evaluate six speech recognizers using metrics including word error rate, latency, and the number of updates to already recognized words on English test data, as well as propose and compare two methods for streaming audio into recognizers for incremental recognition. We further propose Revokes per Second as a new metric for evaluating incremental recognition and demonstrate that it provides insights into overall model performance. We find that, generally, local recognizers are faster and require fewer updates than cloud-based recognizers. Finally, we find Meta's Wav2Vec model to be the fastest, and find Mozilla's DeepSpeech model to be the most stable in its predictions.