# High-precision medical speech recognition through synthetic data and semantic correction: UNITED-MEDASR

arXiv: https://arxiv.org/abs/2412.00055 · HTML: https://mlsanity.com/paper/2412.00055/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 24 Nov 2024.

## Builds on (most cited first, 23 of 23)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training](https://mlsanity.com/paper/2108.06209/) (2021)
- [Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2010.10504/) (2020)
- [StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models](https://mlsanity.com/paper/2306.07691/) (2023)
- [SeamlessM4T: Massively Multilingual & Multimodal Machine Translation](https://mlsanity.com/paper/2308.11596/) (2023)
- [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://mlsanity.com/paper/2305.05084/) (2023)
- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Self-training and Pre-training are Complementary for Speech Recognition](https://mlsanity.com/paper/2010.11430/) (2020)
- [SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network](https://mlsanity.com/paper/2104.02133/) (2021)
- [Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems](https://mlsanity.com/paper/1912.09257/) (2019)
- [HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models](https://mlsanity.com/paper/2309.15701/) (2023)
- [SynthASR: Unlocking Synthetic Data for Speech Recognition](https://mlsanity.com/paper/2106.07803/) (2021)
- [Generative AI for Synthetic Data Across Multiple Medical Modalities: A Systematic Review of Recent Developments and Challenges](https://mlsanity.com/paper/2407.00116/) (2024)
- [ArchiSound: Audio Generation with Diffusion](https://mlsanity.com/paper/2301.13267/) (2023)
- [FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information](https://mlsanity.com/paper/2405.12807/) (2024)
- [Whisper: Fast Flooding for Low-Power Wireless Networks](https://mlsanity.com/paper/1809.03699/) (2018)
- [Mamba for Streaming ASR Combined with Unimodal Aggregation](https://mlsanity.com/paper/2410.00070/) (2024)
- [Deep Generative Models, Synthetic Tabular Data, and Differential Privacy: An Overview and Synthesis](https://mlsanity.com/paper/2307.15424/) (2023)
- [Voice EHR: Introducing Multimodal Audio Data for Health](https://mlsanity.com/paper/2404.01620/) (2024)
- [Assessing Latency in ASR Systems: A Methodological Perspective for Real-Time Use](https://mlsanity.com/paper/2409.05674/) (2024)
- [Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking](https://mlsanity.com/paper/2312.01842/) (2023)

## Built on by (most cited first, 2 of 2)

- [First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI](https://mlsanity.com/paper/2412.09263/) (2024)
- [Inclusivity of AI Speech in Healthcare: A Decade Look Back](https://mlsanity.com/paper/2505.10596/) (2025)

## Abstract (arXiv)

Automatic Speech Recognition (ASR) systems in the clinical domain face significant challenges, notably the need to recognise specialised medical vocabulary accurately and meet stringent precision requirements. We introduce United-MedASR, a novel architecture that addresses these challenges by integrating synthetic data generation, precision ASR fine-tuning, and advanced semantic enhancement techniques. United-MedASR constructs a specialised medical vocabulary by synthesising data from authoritative sources such as ICD-10 (International Classification of Diseases, 10th Revision), MIMS (Monthly Index of Medical Specialties), and FDA databases. This enriched vocabulary helps finetune the Whisper ASR model to better cater to clinical needs. To enhance processing speed, we incorporate Faster Whisper, ensuring streamlined and high-speed ASR performance. Additionally, we employ a customised BART-based semantic enhancer to handle intricate medical terminology, thereby increasing accuracy efficiently. Our layered approach establishes new benchmarks in ASR performance, achieving a Word Error Rate (WER) of 0.985% on LibriSpeech test-clean, 0.26% on Europarl-ASR EN Guest-test, and demonstrating robust performance on Tedlium (0.29% WER) and FLEURS (0.336% WER). Furthermore, we present an adaptable architecture that can be replicated across different domains, making it a versatile solution for domain-specific ASR systems.