# Comparing Large Language Models and Traditional Machine Translation Tools for Translating Medical Consultation Summaries: A Pilot Study

arXiv: https://arxiv.org/abs/2504.16601 · HTML: https://mlsanity.com/paper/2504.16601/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 34 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Builds on (most cited first, 9 of 9)

- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [The unreasonable effectiveness of few-shot learning for machine translation](https://mlsanity.com/paper/2302.01398/) (2023)
- [Document-Level Machine Translation with Large Language Models](https://mlsanity.com/paper/2304.02210/) (2023)
- [Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models](https://mlsanity.com/paper/2403.02715/) (2024)
- [Enriching Biomedical Knowledge for Low-resource Language Through Large-Scale Translation](https://mlsanity.com/paper/2210.05598/) (2022)
- [Instruction-tuned Large Language Models for Machine Translation in the Medical Domain](https://mlsanity.com/paper/2408.16440/) (2024)
- [Towards Accurate Translation via Semantically Appropriate Application of Lexical Constraints](https://mlsanity.com/paper/2306.12089/) (2023)
- [Domain-Specific Translation with Open-Source Large Language Models: Resource-Oriented Analysis](https://mlsanity.com/paper/2412.05862/) (2024)

## Built on by (most cited first, 1 of 1)

- [Multilingual LLM Prompting Strategies for Medical English-Vietnamese Machine Translation](https://mlsanity.com/paper/2509.15640/) (2025)

## Abstract (arXiv)

This study evaluates how well large language models (LLMs) and traditional machine translation (MT) tools translate medical consultation summaries from English into Arabic, Chinese, and Vietnamese. It assesses both patient, friendly and clinician, focused texts using standard automated metrics. Results showed that traditional MT tools generally performed better, especially for complex texts, while LLMs showed promise, particularly in Vietnamese and Chinese, when translating simpler summaries. Arabic translations improved with complexity due to the language's morphology. Overall, while LLMs offer contextual flexibility, they remain inconsistent, and current evaluation metrics fail to capture clinical relevance. The study highlights the need for domain-specific training, improved evaluation methods, and human oversight in medical translation.