# Medicine on the Edge: Comparative Performance Analysis of On-Device LLMs for Clinical Reasoning

arXiv: https://arxiv.org/abs/2502.08954 · HTML: https://mlsanity.com/paper/2502.08954/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 13 Feb 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 9 of 9)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [LLM in a flash: Efficient Large Language Model Inference with Limited Memory](https://mlsanity.com/paper/2312.11514/) (2023)
- [Med42-v2: A Suite of Clinical LLMs](https://mlsanity.com/paper/2408.06142/) (2024)
- [Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities](https://mlsanity.com/paper/2501.19012/) (2025)
- [PrivateLoRA For Efficient Privacy Preserving LLM](https://mlsanity.com/paper/2311.14030/) (2023)
- [A Performance Evaluation of a Quantized Large Language Model on Various Smartphones](https://mlsanity.com/paper/2312.12472/) (2023)

## Built on by (most cited first, 3 of 3)

- [ECG-Agent: On-Device Tool-Calling Agent for ECG Multi-Turn Dialogue](https://mlsanity.com/paper/2601.20323/) (2026)
- [Dual-stage and Lightweight Patient Chart Summarization for Emergency Physicians](https://mlsanity.com/paper/2510.06263/) (2025)
- [On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health](https://mlsanity.com/paper/2609.11961/) (2026)

## Abstract (arXiv)

The deployment of Large Language Models (LLM) on mobile devices offers significant potential for medical applications, enhancing privacy, security, and cost-efficiency by eliminating reliance on cloud-based services and keeping sensitive health data local. However, the performance and accuracy of on-device LLMs in real-world medical contexts remain underexplored. In this study, we benchmark publicly available on-device LLMs using the AMEGA dataset, evaluating accuracy, computational efficiency, and thermal limitation across various mobile devices. Our results indicate that compact general-purpose models like Phi-3 Mini achieve a strong balance between speed and accuracy, while medically fine-tuned models such as Med42 and Aloe attain the highest accuracy. Notably, deploying LLMs on older devices remains feasible, with memory constraints posing a greater challenge than raw processing power. Our study underscores the potential of on-device LLMs for healthcare while emphasizing the need for more efficient inference and models tailored to real-world clinical reasoning.