Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are capable of many natural language tasks, yet they are far from perfect. In health applications, grounding and interpreting domain-specific and non-linguistic data is crucial. This paper investigates the capacity of LLMs to make inferences about health based on contextual information (e.g. user demographics, health knowledge) and physiological data (e.g. resting heart rate, sleep minutes). We present a comprehensive evaluation of 12 state-of-the-art LLMs with prompting and fine-tuning techniques on four public health datasets (PMData, LifeSnaps, GLOBEM and AW_FB). Our experiments cover 10 consumer health prediction tasks in mental health, activity, metabolic, and sleep assessment. Our fine-tuned model, HealthAlpaca exhibits comparable performance to much larger models (GPT-3.5, GPT-4 and Gemini-Pro), achieving the best performance in 8 out of 10 tasks. Ablation studies highlight the effectiveness of context enhancement strategies. Notably, we observe that our context enhancement can yield up to 23.8% improvement in performance. While constructing contextually rich prompts (combining user context, health knowledge and temporal information) exhibits synergistic improvement, the inclusion of health knowledge context in prompts significantly enhances overall performance.
Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Builds on 26 of its 51 references that are in the map. First posted on arXiv 12 Jan 2024.
Builds on
26 of 51 references.
References · 26
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
- 2022Large Language Models Encode Clinical Knowledgecited by 1,830
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
- 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
- 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
- 2022UL2: Unifying Language Learning Paradigmscited by 295
- 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
14 more
- 2023Understanding Social Reasoning in Language Models with Language Modelscited by 172
- 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 163
- 2023Large Language Models are Few-Shot Health Learnerscited by 118
- 2023Towards Accurate Differential Diagnosis with Large Language Modelscited by 116
- 2023Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encodingcited by 71
- 2023Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical Notescited by 51
- 2023Multimodal LLMs for health grounded in individual-specific datacited by 49
- 2023DRG-LLaMA : Tuning LLaMA Model to Predict Diagnosis-related Group for Hospitalized Patientscited by 30
- 2022GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalizationcited by 28
- 2023MindShift: Leveraging Large Language Models for Mental-States-Based Problematic Smartphone Use Interventioncited by 16
- 2020BARD: A structured technique for group elicitation of Bayesian networks to support analytic reasoningcited by 14
- 2023Enhancing Small Medical Learners with Privacy-preserving Contextual Promptingcited by 11
- 2023From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Modelscited by 11
- 2021Personalized Stress Monitoring using Wearable Sensors in Everyday Settingscited by 10
Built on by
Citing papers per year: 2024: 26; 2025: 24; 2026: 16.
66 papers.
Sort:
By area:
- 2024HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?cited by 54
- 2025SensorLM: Learning the Language of Wearable Sensorscited by 49
- 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
- 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
- 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrievalcited by 45
- 2025GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Imagescited by 39
- 2024ALCM: Autonomous LLM-Augmented Causal Discovery Frameworkcited by 29
- 2024Towards a Personal Health Large Language Modelcited by 27
- 2024Text Serialization and Their Relationship with the Conventional Paradigms of Tabular Machine Learningcited by 17
- 2025Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Deliverycited by 14
- 2025FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRAcited by 13
- 2025Edge-First Language Model Inference: Models, Metrics, and Tradeoffscited by 13
38 more
- 2024LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Datacited by 11
- 2024The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognitioncited by 10
- 2025Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoringcited by 10
- 2024Digital Diagnostics: The Potential Of Large Language Models In Recognizing Symptoms Of Common Illnessescited by 8
- 2025Generative AI in Transportation Planning: A Surveycited by 8
- 2025Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safetycited by 8
- 2024Enabling On-Device LLMs Personalization with Smartphone Sensingcited by 7
- 2024MindGuard: Towards Accessible and Sitgma-free Mental Health First Aid via Edge LLMcited by 7
- 2025Time2Lang: Bridging Time-Series Foundation Models and Large Language Models for Health Sensing Beyond Promptingcited by 7
- 2024Quantifying Prediction Consistency Under Fine-Tuning Multiplicity in Tabular LLMscited by 6
- 2024Efficient and Personalized Mobile Health Event Prediction via Small Language Modelscited by 6
- 2025Automated Survey Collection with LLM-based Conversational Agentscited by 6
- 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Datacited by 6
- 2024Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLMcited by 5
- 2024A Demonstration of Adaptive Collaboration of Large Language Models for Medical Decision-Makingcited by 5
- 2025Using LLMs for Late Multimodal Sensor Fusion for Activity Recognitioncited by 5
- 2026ComBodied Agents: a New Paradigm of Human-Centric Agentic AIcited by 5
- 2024Exploration of LLMs, EEG, and behavioral data to measure and support attention and sleepcited by 4
- 2025An LLM-Powered Agent for Physiological Data Analysis: A Case Study on PPG-based Heart Rate Estimationcited by 4
- 2024Large Language Models Memorize Sensor Datasets! Implications on Human Activity Recognition Researchcited by 3
- 2024Graph-Augmented LLMs for Personalized Health Insights: A Case Study in Sleep Analysiscited by 3
- 2024Large Language Models for Cuffless Blood Pressure Measurement From Wearable Biosignalscited by 3
- 2024Copiloting Diagnosis of Autism in Real Clinical Scenarios via LLMscited by 3
- 2025Large Language models for Time Series Analysis: Techniques, Applications, and Challengescited by 3
- 2025HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoringcited by 3
- 2026Wearable Foundation Models Should Go Beyond Static Encoderscited by 3
- 2024Are You Being Tracked? Discover the Power of Zero-Shot Trajectory Tracing with LLMs!cited by 2
- 2024A Short Survey of Human Mobility Prediction in Epidemic Modeling from Transformers to LLMscited by 2
- 2025VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluationcited by 2
- 2025ArtPerception: ASCII Art-based Jailbreak on LLMs with Recognition Pre-testcited by 2
- 2026WEQA: Wearable hEalth Question Answering with Query-Adaptive Agentic Reasoningcited by 2
- 2025Developing Conversational Speech Systems for Robots to Detect Speech Biomarkers of Cognition in People Living with Dementiacited by 1
- 2025RHealthTwin: Towards Responsible and Multimodal Digital Twins for Personalized Well-beingcited by 1
- 2025Predicting Early-Onset Colorectal Cancer with Large Language Modelscited by 1
- 2025Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Studycited by 1
- 2025SePA: A Search-enhanced Predictive Agent for Personalized Health Coachingcited by 1
- 2025Large language models surpass domain-specific architectures for antepartum electronic fetal monitoring analysiscited by 1
- 2026GLIMPSE : Real-Time Text Recognition and Contextual Understanding for VQA in Wearablescited by 1
50 most cited of 66 All 66 (JSON)
- 2026Lapras: Latent Reasoning for Time Series Language Models8 Oct 2026
- 2026Large language models in medical time series analysis7 Sep 2026
- 2026WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data4 Sep 2026
- 2026On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health11 Aug 2026
- 2026ComBodied Agents: a New Paradigm of Human-Centric Agentic AI11 Aug 2026
- 2026WEQA: Wearable hEalth Question Answering with Query-Adaptive Agentic Reasoning16 Jun 2026
- 2026Talking to Your Data: Exploring Embodied Conversation as an Interface for Personal Health Reflection16 Jun 2026
- 2026LLM-Powered Personalized Glycemic Assessment in Type 2 Diabetes with Wearable Sensor Data10 Jun 2026
- 2026From 'What' to 'How' and 'Why': Sharing LLM-Generated Retrospective Summaries of Older Adults' Passive Tracking Data with Remote Family Members2 Jun 2026
- 2026AgensFlow: A Coordination-Policy Substrate for Multi-Agent Systems26 May 2026
- 2026ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?1 May 2026
- 2026An AI Co-Data-Scientist for Prioritizing Candidate Biomarkers from Wearable Sensor Data16 Apr 2026
38 more
- 2026Wearable Foundation Models Should Go Beyond Static Encoders20 Mar 2026
- 2026GLIMPSE : Real-Time Text Recognition and Contextual Understanding for VQA in Wearables13 Feb 2026
- 2026From Knowledge to Inference: Formalizing Specialized Public Health Reasoning on GlobalHealthAtlas31 Jan 2026
- 2026MobiDiary: Autoregressive Action Captioning with Wearable Devices and Wireless Signals13 Jan 2026
- 2025ArtPerception: ASCII Art-based Jailbreak on LLMs with Recognition Pre-test11 Oct 2025
- 2025Using LLMs for Late Multimodal Sensor Fusion for Activity Recognition12 Sep 2025
- 2025Large language models surpass domain-specific architectures for antepartum electronic fetal monitoring analysis9 Sep 2025
- 2025HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring8 Sep 2025
- 2025SePA: A Search-enhanced Predictive Agent for Personalized Health Coaching5 Sep 2025
- 2025Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring12 Aug 2025
- 2025Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Study16 Jun 2025
- 2025Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety14 Jun 2025
- 2025Predicting Early-Onset Colorectal Cancer with Large Language Models13 Jun 2025
- 2025SensorLM: Learning the Language of Wearable Sensors10 Jun 2025
- 2025RHealthTwin: Towards Responsible and Multimodal Digital Twins for Personalized Well-being10 Jun 2025
- 2025Edge-First Language Model Inference: Models, Metrics, and Tradeoffs22 May 2025
- 2025Large Language models for Time Series Analysis: Techniques, Applications, and Challenges21 May 2025
- 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data20 May 2025
- 2025VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluation19 May 2025
- 2025Automated Survey Collection with LLM-based Conversational Agents2 Apr 2025
- 2025Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery1 Apr 2025
- 2025FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA14 Mar 2025
- 2025Generative AI in Transportation Planning: A Survey10 Mar 2025
- 2025GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images8 Mar 2025
- 2025An LLM-Powered Agent for Physiological Data Analysis: A Case Study on PPG-based Heart Rate Estimation18 Feb 2025
- 2025Developing Conversational Speech Systems for Robots to Detect Speech Biomarkers of Cognition in People Living with Dementia15 Feb 2025
- 2025Time2Lang: Bridging Time-Series Foundation Models and Large Language Models for Health Sensing Beyond Prompting11 Feb 2025
- 2025Can ChatGPT Diagnose Alzheimer's Disease?10 Feb 2025
- 2024A Demonstration of Adaptive Collaboration of Large Language Models for Medical Decision-Making31 Oct 2024
- 2024Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLM18 Oct 2024
- 2024Copiloting Diagnosis of Autism in Real Clinical Scenarios via LLMs8 Oct 2024
- 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval6 Oct 2024
- 2024Efficient and Personalized Mobile Health Event Prediction via Small Language Models17 Sep 2024
- 2024MindGuard: Towards Accessible and Sitgma-free Mental Health First Aid via Edge LLM16 Sep 2024
- 2024Tipta uzmanlik sinavinda (tus) buyuk dil modelleri insanlardan daha mi basarili?22 Aug 2024
- 2024MultiSurf-GPT: Facilitating Context-Aware Reasoning with Large-Scale Language Models for Multimodal Surface Sensing14 Aug 2024
- 2024Towards Holistic Disease Risk Prediction using Small Language Models13 Aug 2024
- 2024Exploration of LLMs, EEG, and behavioral data to measure and support attention and sleep1 Aug 2024
50 newest of 66 All 66 (JSON)
- 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrievalcited by 45
- 2024ALCM: Autonomous LLM-Augmented Causal Discovery Frameworkcited by 29
- 2024LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Datacited by 11
- 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Datacited by 6
- 2026WEQA: Wearable hEalth Question Answering with Query-Adaptive Agentic Reasoningcited by 2
- 2024MultiSurf-GPT: Facilitating Context-Aware Reasoning with Large-Scale Language Models for Multimodal Surface Sensingcited by 0
- 2026From Knowledge to Inference: Formalizing Specialized Public Health Reasoning on GlobalHealthAtlascited by 0
- 2026WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Datacited by 0
- 2026Lapras: Latent Reasoning for Time Series Language Modelscited by 0
- 2026Lapras: Latent Reasoning for Time Series Language Models8 Oct 2026
- 2026WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data4 Sep 2026
- 2026WEQA: Wearable hEalth Question Answering with Query-Adaptive Agentic Reasoning16 Jun 2026
- 2026From Knowledge to Inference: Formalizing Specialized Public Health Reasoning on GlobalHealthAtlas31 Jan 2026
- 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data20 May 2025
- 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval6 Oct 2024
- 2024MultiSurf-GPT: Facilitating Context-Aware Reasoning with Large-Scale Language Models for Multimodal Surface Sensing14 Aug 2024
- 2024LLaSA: A Sensor-Aware LLM for Natural Language Reasoning of Human Activity from IMU Data20 Jun 2024
- 2024ALCM: Autonomous LLM-Augmented Causal Discovery Framework2 May 2024
- 2025Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safetycited by 8
- 2025Automated Survey Collection with LLM-based Conversational Agentscited by 6
- 2026ComBodied Agents: a New Paradigm of Human-Centric Agentic AIcited by 5
- 2025An LLM-Powered Agent for Physiological Data Analysis: A Case Study on PPG-based Heart Rate Estimationcited by 4
- 2026AgensFlow: A Coordination-Policy Substrate for Multi-Agent Systemscited by 0
- 2026ComBodied Agents: a New Paradigm of Human-Centric Agentic AI11 Aug 2026
- 2026AgensFlow: A Coordination-Policy Substrate for Multi-Agent Systems26 May 2026
- 2025Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety14 Jun 2025
- 2025Automated Survey Collection with LLM-based Conversational Agents2 Apr 2025
- 2025An LLM-Powered Agent for Physiological Data Analysis: A Case Study on PPG-based Heart Rate Estimation18 Feb 2025
- 2024HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?cited by 54
- 2025SensorLM: Learning the Language of Wearable Sensorscited by 49
- 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
- 2025GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Imagescited by 39
- 2024Towards a Personal Health Large Language Modelcited by 27
- 2024Text Serialization and Their Relationship with the Conventional Paradigms of Tabular Machine Learningcited by 17
- 2025Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Deliverycited by 14
- 2025Edge-First Language Model Inference: Models, Metrics, and Tradeoffscited by 13
- 2024The Strong Pull of Prior Knowledge in Large Language Models and Its Impact on Emotion Recognitioncited by 10
- 2025Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoringcited by 10
- 2024Digital Diagnostics: The Potential Of Large Language Models In Recognizing Symptoms Of Common Illnessescited by 8
- 2025Generative AI in Transportation Planning: A Surveycited by 8
12 most cited of 48 All 48 (JSON)
- 2026Large language models in medical time series analysis7 Sep 2026
- 2026On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health11 Aug 2026
- 2026Talking to Your Data: Exploring Embodied Conversation as an Interface for Personal Health Reflection16 Jun 2026
- 2026LLM-Powered Personalized Glycemic Assessment in Type 2 Diabetes with Wearable Sensor Data10 Jun 2026
- 2026From 'What' to 'How' and 'Why': Sharing LLM-Generated Retrospective Summaries of Older Adults' Passive Tracking Data with Remote Family Members2 Jun 2026
- 2026ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?1 May 2026
- 2026An AI Co-Data-Scientist for Prioritizing Candidate Biomarkers from Wearable Sensor Data16 Apr 2026
- 2026Wearable Foundation Models Should Go Beyond Static Encoders20 Mar 2026
- 2026GLIMPSE : Real-Time Text Recognition and Contextual Understanding for VQA in Wearables13 Feb 2026
- 2026MobiDiary: Autoregressive Action Captioning with Wearable Devices and Wireless Signals13 Jan 2026
- 2025Using LLMs for Late Multimodal Sensor Fusion for Activity Recognition12 Sep 2025
- 2025Large language models surpass domain-specific architectures for antepartum electronic fetal monitoring analysis9 Sep 2025
12 newest of 48 All 48 (JSON)