# MedMobile: A mobile-sized language model with clinical capabilities

arXiv: https://arxiv.org/abs/2410.09019 · HTML: https://mlsanity.com/paper/2410.09019/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 19 of 19)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [Capabilities of Gemini Models in Medicine](https://mlsanity.com/paper/2404.18416/) (2024)
- [MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval](https://mlsanity.com/paper/2307.00589/) (2023)
- [Towards Accurate Differential Diagnosis with Large Language Models](https://mlsanity.com/paper/2312.00164/) (2023)
- [Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions](https://mlsanity.com/paper/2402.18060/) (2024)
- [Long-Short Transformer: Efficient Transformers for Language and Vision](https://mlsanity.com/paper/2107.02192/) (2021)
- [Small LLMs Are Weak Tool Learners: A Multi-LLM Agent](https://mlsanity.com/paper/2401.07324/) (2024)
- [BM25S: Orders of magnitude faster lexical search via eager sparse scoring](https://mlsanity.com/paper/2407.03618/) (2024)
- [UltraMedical: Building Specialized Generalists in Biomedicine](https://mlsanity.com/paper/2406.03949/) (2024)
- [MELTing point: Mobile Evaluation of Language Transformers](https://mlsanity.com/paper/2403.12844/) (2024)
- [K-QA: A Real-World Medical Q&A Benchmark](https://mlsanity.com/paper/2401.14493/) (2024)
- [Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicine](https://mlsanity.com/paper/2406.02394/) (2024)
- [Variational Open-Domain Question Answering](https://mlsanity.com/paper/2210.06345/) (2022)

## Built on by (most cited first, 3 of 3)

- [Can Generalist Vision Language Models (VLMs) Rival Specialist Medical VLMs? Benchmarking and Strategic Insights](https://mlsanity.com/paper/2506.17337/) (2025)
- [Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks](https://mlsanity.com/paper/2512.01191/) (2025)
- [Gaokerena: A Small Persian Medical Language Model Family](https://mlsanity.com/paper/2608.00932/) (2026)

## Abstract (arXiv)

Language models (LMs) have demonstrated expert-level reasoning and recall abilities in medicine. However, computational costs and privacy concerns are mounting barriers to wide-scale implementation. To address these significant limitations, we introduce a parsimonious adaptation of phi-3-mini, MedMobile, a 3.8 billion parameter LM capable of running on a mobile device, for medical applications. We perform a careful set of pipeline additions and demonstrate that chain of thought, ensembling, and fine-tuning lead to the greatest performance gains, while unexpectedly retrieval augmented generation fails to demonstrate significant improvements. We evaluate the efficiency of our pipeline on the MultiMedQA and MedBullets. We demonstrate that MedMobile scores 75.7% on the MedQA (USMLE), surpassing the passing mark for licensed physicians (~60%) and rivaling scores of models 100 times its size. Across the entirety of the MultiMedQA, MedMobile achieves SOTA performance for models with less than 5B parameters and represents the smallest model to pass the MedQA (USMLE). MedMobile holds promise to democratize access to language models in medicine, bolstering lower compute needs and fast inference speeds. With the ability to combat the biggest barriers to entry for language models in medicine, we hope that MedMobile is a critical step forward in developing clinically relevant language models.