# Towards a Personal Health Large Language Model

arXiv: https://arxiv.org/abs/2406.06474 · HTML: https://mlsanity.com/paper/2406.06474/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 66 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 25 of 25)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Towards Expert-Level Medical Question Answering with Large Language Models](https://mlsanity.com/paper/2305.09617/) (2023)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [Capabilities of Gemini Models in Medicine](https://mlsanity.com/paper/2404.18416/) (2024)
- [Holistic Evaluation of Language Models](https://mlsanity.com/paper/2211.09110/) (2022)
- [Large Language Models are Few-Shot Health Learners](https://mlsanity.com/paper/2305.15525/) (2023)
- [Towards Accurate Differential Diagnosis with Large Language Models](https://mlsanity.com/paper/2312.00164/) (2023)
- [Large-scale Training of Foundation Models for Wearable Biosignals](https://mlsanity.com/paper/2312.05409/) (2023)
- [Advancing Multimodal Medical Capabilities of Gemini](https://mlsanity.com/paper/2405.03162/) (2024)
- [Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data](https://mlsanity.com/paper/2401.06866/) (2024)
- [Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuring](https://mlsanity.com/paper/2310.15461/) (2023)
- [MedBLIP: Bootstrapping Language-Image Pre-training from 3D Medical Images and Texts](https://mlsanity.com/paper/2305.10799/) (2023)
- [ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders](https://mlsanity.com/paper/2308.01317/) (2023)
- [A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine](https://mlsanity.com/paper/2405.08603/) (2024)
- [The Capability of Large Language Models to Measure Psychiatric Functioning](https://mlsanity.com/paper/2308.01834/) (2023)
- [IMBUE: Improving Interpersonal Effectiveness through Simulation and Just-in-time Feedback with Human-Language Model Interaction](https://mlsanity.com/paper/2402.12556/) (2024)

## Built on by (most cited first, 27 of 27)

- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://mlsanity.com/paper/2505.08775/) (2025)
- [Mobile Edge Intelligence for Large Language Models: A Contemporary Survey](https://mlsanity.com/paper/2407.18921/) (2024)
- [SensorLM: Learning the Language of Wearable Sensors](https://mlsanity.com/paper/2506.09108/) (2025)
- [SleepLM: Natural-Language Intelligence for Human Sleep](https://mlsanity.com/paper/2602.23605/) (2026)
- [LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data](https://mlsanity.com/paper/2406.14498/) (2024)
- [Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety](https://mlsanity.com/paper/2506.12482/) (2025)
- [Time2Lang: Bridging Time-Series Foundation Models and Large Language Models for Health Sensing Beyond Prompting](https://mlsanity.com/paper/2502.07608/) (2025)
- [ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data](https://mlsanity.com/paper/2505.14038/) (2025)
- [Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLM](https://mlsanity.com/paper/2410.14879/) (2024)
- [Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions](https://mlsanity.com/paper/2601.15353/) (2026)
- [ComBodied Agents: a New Paradigm of Human-Centric Agentic AI](https://mlsanity.com/paper/2608.10915/) (2026)
- [An LLM-Powered Agent for Physiological Data Analysis: A Case Study on PPG-based Heart Rate Estimation](https://mlsanity.com/paper/2502.12836/) (2025)
- [Stabilizing Reasoning in Medical LLMs with Continued Pretraining and Reasoning Preference Optimization](https://mlsanity.com/paper/2504.18080/) (2025)
- [Insulin Resistance Prediction From Wearables and Routine Blood Biomarkers](https://mlsanity.com/paper/2505.03784/) (2025)
- [Graph-Augmented LLMs for Personalized Health Insights: A Case Study in Sleep Analysis](https://mlsanity.com/paper/2406.16252/) (2024)
- [Entry-level guide to the use of large language models for medical research](https://mlsanity.com/paper/2410.18856/) (2024)
- [WEQA: Wearable hEalth Question Answering with Query-Adaptive Agentic Reasoning](https://mlsanity.com/paper/2606.18147/) (2026)
- [OpenMHC: Accelerating the Science of Wearable Foundation Models](https://mlsanity.com/paper/2607.16235/) (2026)
- [SleepCoT: A Lightweight Personalized Sleep Health Model via Chain-of-Thought Distillation](https://mlsanity.com/paper/2410.16924/) (2024)
- [SePA: A Search-enhanced Predictive Agent for Personalized Health Coaching](https://mlsanity.com/paper/2509.04752/) (2025)
- [An AI Co-Data-Scientist for Prioritizing Candidate Biomarkers from Wearable Sensor Data](https://mlsanity.com/paper/2604.14615/) (2026)
- [InvThink: Premortem Reasoning for Safer Language Models](https://mlsanity.com/paper/2510.01569/) (2025)
- [Offline Policy Evaluation of Multi-Turn LLM Health Coaching with Real Users](https://mlsanity.com/paper/2510.17173/) (2025)
- [From 'What' to 'How' and 'Why': Sharing LLM-Generated Retrospective Summaries of Older Adults' Passive Tracking Data with Remote Family Members](https://mlsanity.com/paper/2606.03876/) (2026)
- [Gravity-Aware Hierarchical Routing for Lightweight SensorLLM on Human Activity Recognition](https://mlsanity.com/paper/2606.04019/) (2026)
- [Talking to Your Data: Exploring Embodied Conversation as an Interface for Personal Health Reflection](https://mlsanity.com/paper/2606.17767/) (2026)
- [Large language models in medical time series analysis](https://mlsanity.com/paper/2609.22262/) (2026)

## Abstract (arXiv)

In health, most large language model (LLM) research has focused on clinical tasks. However, mobile and wearable devices, which are rarely integrated into such tasks, provide rich, longitudinal data for personal health monitoring. Here we present Personal Health Large Language Model (PH-LLM), fine-tuned from Gemini for understanding and reasoning over numerical time-series personal health data. We created and curated three datasets that test 1) production of personalized insights and recommendations from sleep patterns, physical activity, and physiological responses, 2) expert domain knowledge, and 3) prediction of self-reported sleep outcomes. For the first task we designed 857 case studies in collaboration with domain experts to assess real-world scenarios in sleep and fitness. Through comprehensive evaluation of domain-specific rubrics, we observed that Gemini Ultra 1.0 and PH-LLM are not statistically different from expert performance in fitness and, while experts remain superior for sleep, fine-tuning PH-LLM provided significant improvements in using relevant domain knowledge and personalizing information for sleep insights. We evaluated PH-LLM domain knowledge using multiple choice sleep medicine and fitness examinations. PH-LLM achieved 79% on sleep and 88% on fitness, exceeding average scores from a sample of human experts. Finally, we trained PH-LLM to predict self-reported sleep quality outcomes from textual and multimodal encoding representations of wearable data, and demonstrate that multimodal encoding is required to match performance of specialized discriminative models. Although further development and evaluation are necessary in the safety-critical personal health domain, these results demonstrate both the broad knowledge and capabilities of Gemini models and the benefit of contextualizing physiological data for personal health applications as done with PH-LLM.