Large Language Models are Few-Shot Health Learners
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) can capture rich representations of concepts that are useful for real-world tasks. However, language alone is limited. While existing LLMs excel at text-based inferences, health applications require that models be grounded in numerical data (e.g., vital signs, laboratory values in clinical domains; steps, movement in the wellness domain) that is not easily or readily expressed as text in existing training corpus. We demonstrate that with only few-shot tuning, a large language model is capable of grounding various physiological and behavioral time-series data and making meaningful inferences on numerous health tasks for both clinical and wellness contexts. Using data from wearable and medical sensor recordings, we evaluate these capabilities on the tasks of cardiac signal analysis, physical activity recognition, metabolic calculation (e.g., calories burned), and estimation of stress reports and mental health screeners.
Key facts. Cited by 115 papers in the map (data through 7 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 24 May 2023.
Builds on
18 of 29 references.
References · 18
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
- 2021Grounded Language-Image Pre-trainingcited by 1,147
- 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
6 more
- 2023How well do Large Language Models perform in Arithmetic tasks?cited by 130
- 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 107
- 2018Grounded Video Descriptioncited by 78
- 2022Limitations of Language Models in Arithmetic and Symbolic Inductioncited by 71
- 2017Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generationcited by 59
- 2021Text-to-Audio Grounding: Building Correspondence Between Captions and Sound Eventscited by 16
Built on by
Citing papers per year: 2023: 15; 2024: 42; 2025: 38; 2026: 20.
115 papers.
Sort:
By area:
- 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
- 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
- 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
- 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
- 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
- 2024Large Language Models for Time Series: A Surveycited by 113
- 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
- 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
- 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
- 2024Language Models Still Struggle to Zero-shot Reason about Time Seriescited by 81
- 2023LLMs Accelerate Annotation for Medical Information Extractioncited by 77
38 more
- 2023Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Datacited by 66
- 2024Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Datacited by 65
- 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
- 2024Empowering Time Series Analysis with Large Language Models: A Surveycited by 60
- 2024Scaling Wearable Foundation Modelscited by 57
- 2024FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusioncited by 53
- 2025SensorLM: Learning the Language of Wearable Sensorscited by 44
- 2023LLMR: Real-time Prompting of Interactive Worlds using Large Language Modelscited by 39
- 2025TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loopcited by 39
- 2024Towards Time Series Reasoning with LLMscited by 36
- 2024SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognitioncited by 35
- 2023The first step is the hardest: Pitfalls of Representing and Tokenizing Temporal Data for Large Language Modelscited by 34
- 2025Context-Alignment: Activating and Enhancing LLM Capabilities in Time Seriescited by 34
- 2025TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agentscited by 33
- 2025OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Datacited by 30
- 2024LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Tracescited by 28
- 2024EEG-GPT: Exploring Capabilities of Large Language Models for EEG Classification and Interpretationcited by 26
- 2023TENT: Connect Language Models with IoT Sensors for Zero-Shot Activity Recognitioncited by 25
- 2024Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learningcited by 25
- 2024Towards a Personal Health Large Language Modelcited by 25
- 2025Multi-modal Time Series Analysis: A Tutorial and Surveycited by 24
- 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23
- 2024Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Datacited by 22
- 2024Mobility-LLM: Learning Visiting Intentions and Travel Preferences from Human Mobility Data with Large Language Modelscited by 21
- 2023Exploring and Characterizing Large Language Models For Embedded System Development and Debuggingcited by 20
- 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
- 2024Chain-of-Interaction: Enhancing Large Language Models for Psychiatric Behavior Understanding by Dyadic Contextscited by 19
- 2024Guided Profile Generation Improves Personalization with LLMscited by 19
- 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
- 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
- 2024Evaluating Large Language Models as Virtual Annotators for Time-series Physical Sensing Datacited by 17
- 2025AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applicationscited by 17
- 2024Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Modelscited by 15
- 2024MedTsLLM: Leveraging LLMs for Multimodal Medical Time Series Analysiscited by 15
- 2024Large Language Models are Zero-Shot Recognizers for Activities of Daily Livingcited by 13
- 2024By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Promptingcited by 12
- 2024Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalizationcited by 12
- 2023From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Modelscited by 11
50 most cited of 115 All 115 (JSON)
- 2026Lapras: Latent Reasoning for Time Series Language Models8 Oct 2026
- 2026Large language models in medical time series analysis7 Sep 2026
- 2026HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care7 Sep 2026
- 2026MnemoDyn: Learning Resting State Dynamics from 40K FMRI sequences25 Aug 2026
- 2026Foundation Models for Epileptogenic Zone Identification in Drug-Resistant Epilepsy21 Jun 2026
- 2026WEQA: Wearable hEalth Question Answering with Query-Adaptive Agentic Reasoning16 Jun 2026
- 2026TiWeaver: Unified Temporal Dynamics Modeling via Contextual Patching2 Jun 2026
- 2026RAPTOR+: A Visually Grounded Vision-Language Framework to Improve Clinical Trust and Auditability in Automated Cancer Referral Processing25 May 2026
- 2026Query-Conditioned Graph Retrieval for Contextualized LLM Reasoning in Personalized Wearable Data10 Apr 2026
- 2026Wearable Foundation Models Should Go Beyond Static Encoders20 Mar 2026
- 2026Learning Transferable Sensor Models via Language-Informed Pretraining12 Mar 2026
- 2026Evaluating Visual Prompts with Eye-Tracking Data for MLLM-Based Human Activity Recognition27 Feb 2026
38 more
- 2026EventCast: Hybrid Demand Forecasting in E-Commerce with LLM-Based Event Knowledge7 Feb 2026
- 2026Multi-scale hypergraph meets LLMs: Aligning large language models for time series analysis4 Feb 2026
- 2026MedConsultBench: A Full-Cycle, Fine-Grained, Process-Aware Benchmark for Medical Consultation Agents19 Jan 2026
- 2026AutoTour: Automatic Photo Tour Guide with Smartphones and LLMs11 Jan 2026
- 2026ConSensus: Multi-Agent Collaboration for Multimodal Sensing10 Jan 2026
- 2026GlyRAG: Context-Aware Retrieval-Augmented Framework for Blood Glucose Forecasting8 Jan 2026
- 2026Prompting Underestimates LLM Capability for Time Series Classification6 Jan 2026
- 2026Rationale-Grounded In-Context Learning for Time Series Reasoning with Multimodal Large Language Models6 Jan 2026
- 2025Training and Evaluation of Guideline-Based Medical Reasoning in LLMs3 Dec 2025
- 2025Aligning Probabilistic Beliefs under Informative Missingness: LLM Steerability in Clinical Reasoning29 Nov 2025
- 2025OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data2 Oct 2025
- 2025Uncovering Zero-Shot Generalization Gaps in Time-Series Foundation Models Using Real-World Videos30 Sep 2025
- 2025Can VLM Pseudo-Labels Train a Time-Series QA Model That Outperforms the VLM?30 Sep 2025
- 2025Integrating Text and Time-Series into (Large) Language Models to Predict Medical Outcomes17 Sep 2025
- 2025Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring12 Aug 2025
- 2025Iterative Learning of Computable Phenotypes for Treatment Resistant Hypertension using Large Language Models7 Aug 2025
- 2025DP-GPT4MTS: Dual-Prompt Large Language Model for Textual-Numerical Time Series Forecasting6 Aug 2025
- 2025Causal Graph Fuzzy LLMs: A First Introduction and Applications in Time Series Forecasting22 Jul 2025
- 2025Mind the XAI Gap: A Human-Centered LLM Framework for Democratizing Explainable AI13 Jun 2025
- 2025SensorLM: Learning the Language of Wearable Sensors10 Jun 2025
- 2025Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review2 Jun 2025
- 2025Few-Shot Optimization for Sensor Data Using Large Language Models: A Case Study on Fatigue Detection24 May 2025
- 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data20 May 2025
- 2025ToolSpectrum : Towards Personalized Tool Utilization for Large Language Models19 May 2025
- 2025HybridServe: Efficient Serving of Large AI Models with Confidence-Based Cascade Routing18 May 2025
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Health13 May 2025
- 2025AI-Generated Fall Data: Assessing LLMs and Diffusion Model for Wearable Fall Detection7 May 2025
- 2025Large Language Models for Drug Overdose Prediction from Longitudinal Medical Records16 Apr 2025
- 2025Enhancing Time Series Forecasting via Multi-Level Text Alignment with LLMs10 Apr 2025
- 2025Right Prediction, Wrong Reasoning: Uncovering LLM Misalignment in RA Disease Diagnosis9 Apr 2025
- 2025Experimental Study on Time Series Analysis of Lower Limb Rehabilitation Exercise Data Driven by Novel Model Architecture and Large Models4 Apr 2025
- 2025Personalized Language Models via Privacy-Preserving Evolutionary Model Merging23 Mar 2025
- 2025Leveraging Large Language Models for Explainable Activity Recognition in Smart Homes: A Critical Evaluation20 Mar 2025
- 2025Multi-modal Time Series Analysis: A Tutorial and Survey17 Mar 2025
- 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook14 Mar 2025
- 2025Deep Learning for Time Series Forecasting: A Survey13 Mar 2025
- 2025AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applications7 Mar 2025
- 2025RAAD-LLM: Adaptive Anomaly Detection Using LLMs and RAG Integration4 Mar 2025
50 newest of 115 All 115 (JSON)
- 2024Language Models Still Struggle to Zero-shot Reason about Time Seriescited by 81
- 2024Towards Time Series Reasoning with LLMscited by 36
- 2025OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Datacited by 30
- 2024LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Tracescited by 28
- 2023From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Modelscited by 11
- 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Datacited by 6
- 2025Right Prediction, Wrong Reasoning: Uncovering LLM Misalignment in RA Disease Diagnosiscited by 4
- 2025Training and Evaluation of Guideline-Based Medical Reasoning in LLMscited by 4
- 2026WEQA: Wearable hEalth Question Answering with Query-Adaptive Agentic Reasoningcited by 2
- 2026Query-Conditioned Graph Retrieval for Contextualized LLM Reasoning in Personalized Wearable Datacited by 1
- 2025Aligning Probabilistic Beliefs under Informative Missingness: LLM Steerability in Clinical Reasoningcited by 0
- 2026Rationale-Grounded In-Context Learning for Time Series Reasoning with Multimodal Large Language Modelscited by 0
12 most cited of 13 All 13 (JSON)
- 2026HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care7 Sep 2026
- 2026WEQA: Wearable hEalth Question Answering with Query-Adaptive Agentic Reasoning16 Jun 2026
- 2026Query-Conditioned Graph Retrieval for Contextualized LLM Reasoning in Personalized Wearable Data10 Apr 2026
- 2026Rationale-Grounded In-Context Learning for Time Series Reasoning with Multimodal Large Language Models6 Jan 2026
- 2025Training and Evaluation of Guideline-Based Medical Reasoning in LLMs3 Dec 2025
- 2025Aligning Probabilistic Beliefs under Informative Missingness: LLM Steerability in Clinical Reasoning29 Nov 2025
- 2025OpenTSLM: Time-Series Language Models for Reasoning over Multivariate Medical Text- and Time-Series Data2 Oct 2025
- 2025ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data20 May 2025
- 2025Right Prediction, Wrong Reasoning: Uncovering LLM Misalignment in RA Disease Diagnosis9 Apr 2025
- 2024Towards Time Series Reasoning with LLMs17 Sep 2024
- 2024Language Models Still Struggle to Zero-shot Reason about Time Series17 Apr 2024
- 2024LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Traces28 Mar 2024
12 newest of 13 All 13 (JSON)
- 2025TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agentscited by 33
- 2025An LLM-Powered Agent for Physiological Data Analysis: A Case Study on PPG-based Heart Rate Estimationcited by 4
- 2026ConSensus: Multi-Agent Collaboration for Multimodal Sensingcited by 3
- 2026ConSensus: Multi-Agent Collaboration for Multimodal Sensing10 Jan 2026
- 2025An LLM-Powered Agent for Physiological Data Analysis: A Case Study on PPG-based Heart Rate Estimation18 Feb 2025
- 2025TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents17 Feb 2025
- 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
- 2024By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Promptingcited by 12
- 2024Enhancing Journalism with AI: A Study of Contextualized Image Captioning for News Articles using LLMs and LMMscited by 1
- 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches6 Jan 2025
- 2024Enhancing Journalism with AI: A Study of Contextualized Image Captioning for News Articles using LLMs and LMMs8 Aug 2024
- 2024By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Prompting15 Jul 2024
- 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
- 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
- 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
- 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
- 2023ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewingcited by 118
- 2024Large Language Models for Time Series: A Surveycited by 113
- 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
- 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
- 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
- 2023LLMs Accelerate Annotation for Medical Information Extractioncited by 77
- 2023Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Datacited by 66
12 most cited of 91 All 91 (JSON)
- 2026Lapras: Latent Reasoning for Time Series Language Models8 Oct 2026
- 2026Large language models in medical time series analysis7 Sep 2026
- 2026MnemoDyn: Learning Resting State Dynamics from 40K FMRI sequences25 Aug 2026
- 2026Foundation Models for Epileptogenic Zone Identification in Drug-Resistant Epilepsy21 Jun 2026
- 2026TiWeaver: Unified Temporal Dynamics Modeling via Contextual Patching2 Jun 2026
- 2026RAPTOR+: A Visually Grounded Vision-Language Framework to Improve Clinical Trust and Auditability in Automated Cancer Referral Processing25 May 2026
- 2026Wearable Foundation Models Should Go Beyond Static Encoders20 Mar 2026
- 2026Learning Transferable Sensor Models via Language-Informed Pretraining12 Mar 2026
- 2026Evaluating Visual Prompts with Eye-Tracking Data for MLLM-Based Human Activity Recognition27 Feb 2026
- 2026EventCast: Hybrid Demand Forecasting in E-Commerce with LLM-Based Event Knowledge7 Feb 2026
- 2026Multi-scale hypergraph meets LLMs: Aligning large language models for time series analysis4 Feb 2026
- 2026MedConsultBench: A Full-Cycle, Fine-Grained, Process-Aware Benchmark for Medical Consultation Agents19 Jan 2026
12 newest of 91 All 91 (JSON)