# Word Error Rate Estimation Without ASR Output: e-WER2

arXiv: https://arxiv.org/abs/2008.03403 · HTML: https://mlsanity.com/paper/2008.03403/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2020.

## Built on by (most cited first, 8 of 8)

- [A Reference-less Quality Metric for Automatic Speech Recognition via Contrastive-Learning of a Multi-Language Model with Self-Supervision](https://mlsanity.com/paper/2306.13114/) (2023)
- [Multilingual Word Error Rate Estimation: e-WER3](https://mlsanity.com/paper/2304.00649/) (2023)
- [NoRefER: a Referenceless Quality Metric for Automatic Speech Recognition via Semi-Supervised Language Model Fine-Tuning with Contrastive Learning](https://mlsanity.com/paper/2306.12577/) (2023)
- [Fast Word Error Rate Estimation Using Self-Supervised Representations for Speech and Text](https://mlsanity.com/paper/2310.08225/) (2023)
- [Big model only for hard audios: Sample dependent Whisper model selection for efficient inferences](https://mlsanity.com/paper/2309.12712/) (2023)
- [Automatic Speech Recognition System-Independent Word Error Rate Estimation](https://mlsanity.com/paper/2404.16743/) (2024)
- [On the Robust Approximation of ASR Metrics](https://mlsanity.com/paper/2502.12408/) (2025)
- [Read What You Hear: Reference-Free Hypotheses Evaluation with Acoustic Discrepancy](https://mlsanity.com/paper/2606.04680/) (2026)

## Abstract (arXiv)

Measuring the performance of automatic speech recognition (ASR) systems requires manually transcribed data in order to compute the word error rate (WER), which is often time-consuming and expensive. In this paper, we continue our effort in estimating WER using acoustic, lexical and phonotactic features. Our novel approach to estimate the WER uses a multistream end-to-end architecture. We report results for systems using internal speech decoder features (glass-box), systems without speech decoder features (black-box), and for systems without having access to the ASR system (no-box). The no-box system learns joint acoustic-lexical representation from phoneme recognition results along with MFCC acoustic features to estimate WER. Considering WER per sentence, our no-box system achieves 0.56 Pearson correlation with the reference evaluation and 0.24 root mean square error (RMSE) across 1,400 sentences. The estimated overall WER by e-WER2 is 30.9% for a three hours test set, while the WER computed using the reference transcriptions was 28.5%.