mlsanityData to 7 Oct 2026

Earnings-22: A Practical Benchmark for Accents in the Wild

arXiv 2203.15591 ↗First posted 29 Mar 2022cs.CLcited by 46

Not on the map. Why?

Abstract (arXiv)

Modern automatic speech recognition (ASR) systems have achieved superhuman Word Error Rate (WER) on many common corpora despite lacking adequate performance on speech in the wild. Beyond that, there is a lack of real-world, accented corpora to properly benchmark academic and commercial models. To ensure this type of speech is represented in ASR benchmarking, we present Earnings-22, a 125 file, 119 hour corpus of English-language earnings calls gathered from global companies. We run a comparison across 4 commercial models showing the variation in performance when taking country of origin into consideration. Looking at hypothesis transcriptions, we explore errors common to all ASR systems tested. By examining Individual Word Error Rate (IWER), we find that key speech features impact model performance more for certain accents than others. Earnings-22 provides a free-to-use benchmark of real-world, accented audio to bridge academic and industrial research.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 2 of its 12 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

2 of 12 references.

References · 2

  1. 2021SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognitioncited by 77
  2. 2021Earnings-21: A Practical Benchmark for ASR in the Wildcited by 60

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 9; 2025: 12; 2026: 18.

46 papers.

Sort:

By area:

  1. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 124
  2. 2023Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labellingcited by 87
  3. 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 64
  4. 2025Voxtralcited by 41
  5. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 24
  6. 2025Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluationcited by 21
  7. 2023Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognitioncited by 20
  8. 2022ESB: A Benchmark For Multi-Domain End-to-End Speech Recognitioncited by 18
  9. 2023Zero-shot Domain-sensitive Speech Recognition with Prompt-conditioning Fine-tuningcited by 13
  10. 2023Updated Corpora and Benchmarks for Long-Form Speech Recognitioncited by 10
  11. 2025Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyondcited by 10
  12. 2024NeKo: Cross-Modality Post-Recognition Error Correction with Tasks-Guided Mixture-of-Experts Language Modelcited by 7
34 more
  1. 2024MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Modelscited by 7
  2. 2025Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Modelscited by 5
  3. 2026Voxtral Realtimecited by 5
  4. 2024Longer is (Not Necessarily) Stronger: Punctuated Long-Sequence Training for Enhanced Speech Recognition and Translationcited by 4
  5. 2025WhisperKit: On-device Real-time ASR with Billion-Scale Transformerscited by 4
  6. 2023How Much Context Does My Attention-Based ASR System Need?cited by 3
  7. 2025Drax: Speech Recognition with Discrete Flow Matchingcited by 3
  8. 2025PROFASR-BENCH: A Benchmark for Context-Conditioned ASR in High-Stakes Professional Speechcited by 3
  9. 2026Whisper-CD: Accurate Long-Form Speech Recognition using Multi-Negative Contrastive Decodingcited by 3
  10. 2026IndicContextEval: A Benchmark for Evaluating Context Utilisation in Audio Large Language Models Across 8 Indic Languagescited by 3
  11. 2024Beyond Levenshtein: Leveraging Multiple Algorithms for Robust Word Error Rate Computations And Granular Error Classificationscited by 2
  12. 2025On the Robust Approximation of ASR Metricscited by 2
  13. 2025FinAudio: A Benchmark for Audio Large Language Models in Financial Applicationscited by 2
  14. 2025SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Modelscited by 2
  15. 2026Contextual Earnings-22: A Speech Recognition Benchmark with Custom Vocabulary in the Wildcited by 2
  16. 2026Hijacking Large Audio-Language Models via Context-Agnostic and Imperceptible Auditory Prompt Injectioncited by 2
  17. 2023Robust Automatic Speech Recognition via WavAugment Guided Phoneme Adversarial Trainingcited by 1
  18. 2024Unifying Global and Near-Context Biasing in a Single Trie Passcited by 1
  19. 2025Whisfusion: Parallel ASR Decoding with Masked Diffusioncited by 1
  20. 2026ASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognitioncited by 1
  21. 2026ReHear: Iterative Pseudo-Label Refinement for Semi-Supervised Speech Recognition via Audio Large Language Modelscited by 1
  22. 2026From Text Metrics to Model Internals: A Study of Whisper ASR Hallucination Detectioncited by 1
  23. 2026Preference-ASR: A Preference-Aware Test Set for Benchmarking ASR in the Era of Speech LLMscited by 1
  24. 2024Self-Train Before You Transcribecited by 0
  25. 2024Less is More: Accurate Speech Recognition & Translation without Web-Scale Datacited by 0
  26. 2026FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptationcited by 0
  27. 2026TRADE: Transducer-Augmented Decoder for Speech LLMcited by 0
  28. 2026HALAS: A Human-Annotated Dataset of Hallucinations of Modern ASR Systemscited by 0
  29. 2026Earnings25: A Comprehensive 500-Hour Speech Benchmark for Financecited by 0
  30. 2026StreamHear: Domain-Adapted Pseudo-Labeling for Semi-Supervised Streaming Speech Recognitioncited by 0
  31. 2026The Null Token Knows: Reducing Message-Free Hallucination in ASR and NMTcited by 0
  32. 2026Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactionscited by 0
  33. 2026Design of the IBM Granite 5.0 TurboCTC ASR Modelcited by 0
  34. 2026How to Reduce Whisper Hallucinationcited by 0