mlsanityData to 14 Oct 2026

Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond

arXiv 2502.03945 ↗First posted 6 Feb 2025cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Speech technologies are transforming interactions across various sectors, from healthcare to call centers and robots, yet their performance on African-accented conversations remains underexplored. We introduce Afrispeech-Dialog, a benchmark dataset of 50 simulated medical and non-medical African-accented English conversations, designed to evaluate automatic speech recognition (ASR) and related technologies. We assess state-of-the-art (SOTA) speaker diarization and ASR systems on long-form, accented speech, comparing their performance with native accents and discover a 10%+ performance degradation. Additionally, we explore medical conversation summarization capabilities of large language models (LLMs) to demonstrate the impact of ASR errors on downstream medical summaries, providing insights into the challenges and opportunities for speech technologies in the Global South. Our work highlights the need for more inclusive datasets to advance conversational AI in low-resource settings.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 37 of its 70 references that are in the map. First posted on arXiv 6 Feb 2025.

Builds on

37 of 70 references.

References · 37

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  9. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  10. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  11. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  12. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
25 more
  1. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  2. 2019pyannote.audio: neural building blocks for speaker diarizationcited by 181
  3. 2018Speaker Recognition from Raw Waveform with SincNetcited by 166
  4. 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 118
  5. 2015Yara Parser: A Fast and Accurate Dependency Parsercited by 111
  6. 2023Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labellingcited by 90
  7. 2024Med42-v2: A Suite of Clinical LLMscited by 90
  8. 2020Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard taskscited by 73
  9. 2023Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarizationcited by 69
  10. 2022Earnings-22: A Practical Benchmark for Accents in the Wildcited by 50
  11. 2023AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASRcited by 35
  12. 2024MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applicationscited by 25
  13. 2024Intelligent Clinical Documentation: Harnessing Generative AI for Patient-Centric Clinical Note Generationcited by 20
  14. 2023Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluationscited by 18
  15. 2022Accented Speech Recognition: Benchmarking, Pre-training, and Diverse Datacited by 15
  16. 2018Building a Unified Code-Switching ASR System for South African Languagescited by 12
  17. 2021Accented Speech Recognition: A Surveycited by 12
  18. 2023MERLIon CCS Challenge: A English-Mandarin code-switching child-directed speech corpus for language identification and diarizationcited by 9
  19. 20241000 African Voices: Advancing inclusive multi-speaker multi-accent speech synthesiscited by 9
  20. 2024EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluationcited by 8
  21. 2023AfriNames: Most ASR models "butcher" African Namescited by 7
  22. 2024Performant ASR Models for Medical Entities in Accented Speechcited by 7
  23. 2022EVI: Multilingual Spoken Dialogue Tasks and Dataset for Knowledge-Based Enrolment, Verification, and Identificationcited by 2
  24. 2023What We Know So Far: Artificial Intelligence in African Healthcarecited by 2
  25. 2024Real-time Speech Summarization for Medical Conversationscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 8.

11 papers.

  1. 2026UbuntuGuard: A Culturally-Grounded Policy Benchmark for Equitable AI Safety in African Languagescited by 5
  2. 2026The Persona Paradox: Medical Personas as Behavioral Priors in Clinical Language Modelscited by 3
  3. 2025Domain-Aware Speaker Diarization On African-Accented Englishcited by 2
  4. 2025Automatic Speech Recognition (ASR) for African Low-Resource Languages: A Systematic Literature Reviewcited by 2
  5. 2025Can large audio language models understand child stuttering speech? speech summarization, and source separationcited by 2
  6. 2026Afrispeech Semantics: Evaluating Audio Semantic Reasoning in Spoken Language Models Across Domains and Accentscited by 2
  7. 2026Reasoning Beyond Transcription: Audio Language Models on Child Stuttering Speechcited by 1
  8. 2026AfriVox-v2: A Domain-Verticalized Benchmark for In-the-Wild African Speech Recognitioncited by 0
  9. 2026Beyond Word Error Rate: A Switch Aware Evaluation of ASR and Audio Language Models on English Yoruba Code-Switched Speechcited by 0
  10. 2026Neyshekar: An Open Persian Read-Speech Corpus for Automatic Speech Recognitioncited by 0
  11. 2026How Robust Are Neural Audio Codecs for African Speech? A Multi-Task Benchmark and the Limits of Perceptual Qualitycited by 0