# HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?

arXiv: https://arxiv.org/abs/2403.02727 · HTML: https://mlsanity.com/paper/2403.02727/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 5 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security](https://mlsanity.com/paper/2401.05459/) (2024)
- [ChatGPT: Jack of all trades, master of none](https://mlsanity.com/paper/2302.10724/) (2023)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data](https://mlsanity.com/paper/2401.06866/) (2024)
- [Penetrative AI: Making LLMs Comprehend the Physical World](https://mlsanity.com/paper/2310.09605/) (2023)
- [When Large Language Model Agents Meet 6G Networks: Perception, Grounding, and Alignment](https://mlsanity.com/paper/2401.07764/) (2024)

## Built on by (most cited first, 30 of 54)

- [SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognition](https://mlsanity.com/paper/2410.10624/) (2024)
- [UniMTS: Unified Pre-training for Motion Time Series](https://mlsanity.com/paper/2410.19818/) (2024)
- [DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledge](https://mlsanity.com/paper/2405.12541/) (2024)
- [IoT-LLM: a framework for enhancing Large Language Model reasoning from real-world sensor data](https://mlsanity.com/paper/2410.02429/) (2024)
- [AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applications](https://mlsanity.com/paper/2503.05346/) (2025)
- [ZARA: Training-Free Motion Time-Series Reasoning via Evidence-Grounded LLM Agents](https://mlsanity.com/paper/2508.04038/) (2025)
- [Layout Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST)](https://mlsanity.com/paper/2405.12368/) (2024)
- [Large Language Models are Zero-Shot Recognizers for Activities of Daily Living](https://mlsanity.com/paper/2407.01238/) (2024)
- [SensorChat: Answering Qualitative and Quantitative Questions during Long-Term Multimodal Sensor Interactions](https://mlsanity.com/paper/2502.02883/) (2025)
- [ProAgent: Harnessing On-Demand Sensory Contexts for Proactive LLM Agent Systems in the Wild](https://mlsanity.com/paper/2512.06721/) (2025)
- [Using Large Language Models to Compare Explainable Models for Smart Home Human Activity Recognition](https://mlsanity.com/paper/2408.06352/) (2024)
- [MindGuard: Towards Accessible and Sitgma-free Mental Health First Aid via Edge LLM](https://mlsanity.com/paper/2409.10064/) (2024)
- [RAG-HAR: Retrieval Augmented Generation-based Human Activity Recognition](https://mlsanity.com/paper/2512.08984/) (2025)
- [Large Language Model-Guided Semantic Alignment for Human Activity Recognition](https://mlsanity.com/paper/2410.00003/) (2024)
- [EmbedGenius: Towards Automated Software Development for Generic Embedded IoT Systems](https://mlsanity.com/paper/2412.09058/) (2024)
- [Wi-Chat: Large Language Model Powered Wi-Fi Sensing](https://mlsanity.com/paper/2502.12421/) (2025)
- [ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data](https://mlsanity.com/paper/2505.14038/) (2025)
- [Towards Generalizable Human Activity Recognition: A Survey](https://mlsanity.com/paper/2508.12213/) (2025)
- [LightLLM: A Versatile Large Language Model for Predictive Light Sensing](https://mlsanity.com/paper/2411.15211/) (2024)
- [Leveraging Large Language Models for Explainable Activity Recognition in Smart Homes: A Critical Evaluation](https://mlsanity.com/paper/2503.16622/) (2025)
- [A Survey of Foundation Models for IoT: Taxonomy and Criteria-Based Analysis](https://mlsanity.com/paper/2506.12263/) (2025)
- [DailyLLM: Context-Aware Activity Log Generation Using Multi-Modal Sensors and LLMs](https://mlsanity.com/paper/2507.13737/) (2025)
- [QuAnTS: Question Answering on Time Series](https://mlsanity.com/paper/2511.05124/) (2025)
- [TransCompressor: LLM-Powered Multimodal Data Compression for Smart Transportation](https://mlsanity.com/paper/2411.16020/) (2024)
- [Large Language Models Memorize Sensor Datasets! Implications on Human Activity Recognition Research](https://mlsanity.com/paper/2406.05900/) (2024)
- [Are You Being Tracked? Discover the Power of Zero-Shot Trajectory Tracing with LLMs!](https://mlsanity.com/paper/2403.06201/) (2024)
- [SensorBench: Benchmarking LLMs in Coding-Based Sensor Processing](https://mlsanity.com/paper/2410.10741/) (2024)
- [MHARFedLLM: Multimodal Human Activity Recognition Using Federated Large Language Model](https://mlsanity.com/paper/2508.01701/) (2025)
- [RAG-HAR+: Towards Cost-Efficient LLM-Based Human Activity Recognition for Edge Deployment](https://mlsanity.com/paper/2607.26631/) (2026)
- [Human Re-ID Meets LVLMs: What can we expect?](https://mlsanity.com/paper/2501.18698/) (2025)

## Abstract (arXiv)

There is an ongoing debate regarding the potential of Large Language Models (LLMs) as foundational models seamlessly integrated with Cyber-Physical Systems (CPS) for interpreting the physical world. In this paper, we carry out a case study to answer the following question: Are LLMs capable of zero-shot human activity recognition (HAR). Our study, HARGPT, presents an affirmative answer by demonstrating that LLMs can comprehend raw IMU data and perform HAR tasks in a zero-shot manner, with only appropriate prompts. HARGPT inputs raw IMU data into LLMs and utilizes the role-play and think step-by-step strategies for prompting. We benchmark HARGPT on GPT4 using two public datasets of different inter-class similarities and compare various baselines both based on traditional machine learning and state-of-the-art deep classification models. Remarkably, LLMs successfully recognize human activities from raw IMU data and consistently outperform all the baselines on both datasets. Our findings indicate that by effective prompting, LLMs can interpret raw IMU data based on their knowledge base, possessing a promising potential to analyze raw sensor data of the physical world effectively.