# HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring

arXiv: https://arxiv.org/abs/2509.07260 · HTML: https://mlsanity.com/paper/2509.07260/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 62 references that are in the map.
- First posted on arXiv 8 Sep 2025.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data](https://mlsanity.com/paper/2307.14385/) (2023)
- [Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data](https://mlsanity.com/paper/2401.06866/) (2024)
- [Privacy in Large Language Models: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2310.10383/) (2023)
- [PhysioLLM: Supporting Personalized Health Insights with Wearables and Large Language Models](https://mlsanity.com/paper/2406.19283/) (2024)
- [MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases](https://mlsanity.com/paper/2406.10290/) (2024)
- [Camel: Energy-Aware LLM Inference on Resource-Constrained Devices](https://mlsanity.com/paper/2508.09173/) (2025)
- [Exploring Large-Scale Language Models to Evaluate EEG-Based Multimodal Data for Mental Health](https://mlsanity.com/paper/2408.07313/) (2024)
- [DeepHeart: Semi-Supervised Sequence Learning for Cardiovascular Risk Prediction](https://mlsanity.com/paper/1802.02511/) (2018)
- [Efficient and Personalized Mobile Health Event Prediction via Small Language Models](https://mlsanity.com/paper/2409.18987/) (2024)
- [Beyond Scale: Small Language Models are Comparable to GPT-4 in Mental Health Understanding](https://mlsanity.com/paper/2507.08031/) (2025)
- [Learning Individualized Cardiovascular Responses from Large-scale Wearable Sensors Data](https://mlsanity.com/paper/1812.01696/) (2018)
- [UR2M: Uncertainty and Resource-Aware Event Detection on Microcontrollers](https://mlsanity.com/paper/2402.09264/) (2024)
- [UDAMA: Unsupervised Domain Adaptation through Multi-discriminator Adversarial Training with Noisy Labels Improves Cardio-fitness Prediction](https://mlsanity.com/paper/2307.16651/) (2023)
- [Small Language Models (SLMs) Can Still Pack a Punch: A survey (updated 2026)](https://mlsanity.com/paper/2501.05465/) (2025)

## Built on by (most cited first, 3 of 3)

- [Menta: A Small Language Model for On-Device Mental Health Prediction](https://mlsanity.com/paper/2512.02716/) (2025)
- [Pocket-Dentist: On-Device Dental Image Understanding via Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2605.29299/) (2026)
- [Little Brains, Big Feats: Exploring Compact Language Models](https://mlsanity.com/paper/2606.30062/) (2026)

## Abstract (arXiv)

Mobile and wearable healthcare monitoring play a vital role in facilitating timely interventions, managing chronic health conditions, and ultimately improving individuals' quality of life. Previous studies on large language models (LLMs) have highlighted their impressive generalization abilities and effectiveness in healthcare prediction tasks. However, most LLM-based healthcare solutions are cloud-based, which raises significant privacy concerns and results in increased memory usage and latency. To address these challenges, there is growing interest in compact models, Small Language Models (SLMs), which are lightweight and designed to run locally and efficiently on mobile and wearable devices. Nevertheless, how well these models perform in healthcare prediction remains largely unexplored. We systematically evaluated SLMs on health prediction tasks using zero-shot, few-shot, and instruction fine-tuning approaches, and deployed the best performing fine-tuned SLMs on mobile devices to evaluate their real-world efficiency and predictive performance in practical healthcare scenarios. Our results show that SLMs can achieve performance comparable to LLMs while offering substantial gains in efficiency and privacy. However, challenges remain, particularly in handling class imbalance and few-shot scenarios. These findings highlight SLMs, though imperfect in their current form, as a promising solution for next-generation, privacy-preserving healthcare monitoring.