# The Sound of Healthcare: Improving Medical Transcription ASR Accuracy with Large Language Models

arXiv: https://arxiv.org/abs/2402.07658 · HTML: https://mlsanity.com/paper/2402.07658/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 59 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [AudioPaLM: A Large Language Model That Can Speak and Listen](https://mlsanity.com/paper/2306.12925/) (2023)
- [LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models](https://mlsanity.com/paper/2309.12307/) (2023)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [Better Zero-Shot Reasoning with Role-Play Prompting](https://mlsanity.com/paper/2308.07702/) (2023)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [Prompting Large Language Models with Speech Recognition Abilities](https://mlsanity.com/paper/2307.11795/) (2023)
- [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022)
- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking](https://mlsanity.com/paper/2311.09827/) (2023)
- [Can Generative Large Language Models Perform ASR Error Correction?](https://mlsanity.com/paper/2307.04172/) (2023)
- [Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions](https://mlsanity.com/paper/2307.13339/) (2023)
- [PriMock57: A Dataset Of Primary Care Mock Consultations](https://mlsanity.com/paper/2204.00333/) (2022)
- [Sorting through the noise: Testing robustness of information processing in pre-trained language models](https://mlsanity.com/paper/2109.12393/) (2021)
- [Relating Neural Text Degeneration to Exposure Bias](https://mlsanity.com/paper/2109.08705/) (2021)
- [Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach](https://mlsanity.com/paper/2309.05248/) (2023)
- [Leveraging Large Language Models for Exploiting ASR Uncertainty](https://mlsanity.com/paper/2309.04842/) (2023)
- [A Stable and Effective Learning Strategy for Trainable Greedy Decoding](https://mlsanity.com/paper/1804.07915/) (2018)
- [Retrieval Augmented Generation and Representative Vector Summarization for large unstructured textual data in Medical Education](https://mlsanity.com/paper/2308.00479/) (2023)
- [Integrating Pre-Trained Speech and Language Models for End-to-End Speech Recognition](https://mlsanity.com/paper/2312.03668/) (2023)
- [Global Performance Disparities Between English-Language Accents in Automatic Speech Recognition](https://mlsanity.com/paper/2208.01157/) (2022)

## Built on by (most cited first, 18 of 18)

- [LLM-based speaker diarization correction: A generalizable approach](https://mlsanity.com/paper/2406.04927/) (2024)
- [From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practice](https://mlsanity.com/paper/2410.01812/) (2024)
- [Performant ASR Models for Medical Entities in Accented Speech](https://mlsanity.com/paper/2406.12387/) (2024)
- [AG-LSEC: Audio Grounded Lexical Speaker Error Correction](https://mlsanity.com/paper/2406.17266/) (2024)
- [Enhancing Dialogue Annotation with Speaker Characteristics Leveraging a Frozen LLM](https://mlsanity.com/paper/2508.04795/) (2025)
- [SilVar: Speech Driven Multimodal Model for Reasoning Visual Question Answering and Object Localization](https://mlsanity.com/paper/2412.16771/) (2024)
- [SEAL: Speaker Error Correction using Acoustic-conditioned Large Language Models](https://mlsanity.com/paper/2501.08421/) (2025)
- [Lost in Transcription: How Speech-to-Text Errors Derail Code Understanding](https://mlsanity.com/paper/2601.15339/) (2026)
- [Searching for Best Practices in Medical Transcription with Large Language Model](https://mlsanity.com/paper/2410.03797/) (2024)
- [Preserving Privacy, Increasing Accessibility, and Reducing Cost: An On-Device Artificial Intelligence Model for Medical Transcription and Note Generation](https://mlsanity.com/paper/2507.03033/) (2025)
- [Iterative LLM-based improvement for French Clinical Interview Transcription and Speaker Diarization](https://mlsanity.com/paper/2603.00086/) (2026)
- [Evaluating a Multi-Agent Voice-Enabled Smart Speaker for Care Homes: A Safety-Focused Framework](https://mlsanity.com/paper/2603.23625/) (2026)
- [A Sociolinguistic Analysis of Automatic Speech Recognition Bias in Newcastle English](https://mlsanity.com/paper/2603.24549/) (2026)
- [Evaluating Fine-Tuned LLM Model For Medical Transcription With Small Low-Resource Languages Validated Dataset](https://mlsanity.com/paper/2603.24772/) (2026)
- [A Dataset of Robot-Patient and Doctor-Patient Medical Dialogues for Spoken Language Processing Tasks](https://mlsanity.com/paper/2605.26747/) (2026)
- [Understanding Multilingual Medical ASR Adaptation Through Layer-Wise Analysis](https://mlsanity.com/paper/2608.18825/) (2026)
- [Mitigating Speaker Leakage in Cascaded Multi-talker ASR with Diarization-based Transcript Correction](https://mlsanity.com/paper/2608.22196/) (2026)
- [MedWER: A Reproducible, Model-Free Evaluation Protocol for Medical Speech Recognition](https://mlsanity.com/paper/2609.05728/) (2026)

## Abstract (arXiv)

In the rapidly evolving landscape of medical documentation, transcribing clinical dialogues accurately is increasingly paramount. This study explores the potential of Large Language Models (LLMs) to enhance the accuracy of Automatic Speech Recognition (ASR) systems in medical transcription. Utilizing the PriMock57 dataset, which encompasses a diverse range of primary care consultations, we apply advanced LLMs to refine ASR-generated transcripts. Our research is multifaceted, focusing on improvements in general Word Error Rate (WER), Medical Concept WER (MC-WER) for the accurate transcription of essential medical terms, and speaker diarization accuracy. Additionally, we assess the role of LLM post-processing in improving semantic textual similarity, thereby preserving the contextual integrity of clinical dialogues. Through a series of experiments, we compare the efficacy of zero-shot and Chain-of-Thought (CoT) prompting techniques in enhancing diarization and correction accuracy. Our findings demonstrate that LLMs, particularly through CoT prompting, not only improve the diarization accuracy of existing ASR systems but also achieve state-of-the-art performance in this domain. This improvement extends to more accurately capturing medical concepts and enhancing the overall semantic coherence of the transcribed dialogues. These findings illustrate the dual role of LLMs in augmenting ASR outputs and independently excelling in transcription tasks, holding significant promise for transforming medical ASR systems and leading to more accurate and reliable patient records in healthcare settings.