# Large Language Models Memorize Sensor Datasets! Implications on Human Activity Recognition Research

arXiv: https://arxiv.org/abs/2406.05900 · HTML: https://mlsanity.com/paper/2406.05900/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 9 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Large Language Models for Mathematical Reasoning: Progresses and Challenges](https://mlsanity.com/paper/2402.00157/) (2024)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [Large Language Models for Robotics: A Survey](https://mlsanity.com/paper/2311.07226/) (2023)
- [AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model](https://mlsanity.com/paper/2309.16058/) (2023)
- [Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data](https://mlsanity.com/paper/2401.06866/) (2024)
- [HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?](https://mlsanity.com/paper/2403.02727/) (2024)
- [Controlling the Extraction of Memorized Data from Large Language Models via Prompt-Tuning](https://mlsanity.com/paper/2305.11759/) (2023)
- [Proving membership in LLM pretraining data via data watermarks](https://mlsanity.com/paper/2402.10892/) (2024)
- [Elephants Never Forget: Testing Language Models for Memorization of Tabular Data](https://mlsanity.com/paper/2403.06644/) (2024)
- [Evaluating Large Language Models as Virtual Annotators for Time-series Physical Sensing Data](https://mlsanity.com/paper/2403.01133/) (2024)

## Built on by (most cited first, 3 of 3)

- [HAROOD: A Benchmark for Out-of-distribution Generalization in Sensor-based Human Activity Recognition](https://mlsanity.com/paper/2512.10807/) (2025)
- [Saga: Capturing Multi-granularity Semantics from Massive Unlabelled IMU Data for User Perception](https://mlsanity.com/paper/2504.11726/) (2025)
- [More Features Are Not More Evidence: Limits of Training-Free Human Activity Recognition with Jev](https://mlsanity.com/paper/2609.36154/) (2026)

## Abstract (arXiv)

The astonishing success of Large Language Models (LLMs) in Natural Language Processing (NLP) has spurred their use in many application domains beyond text analysis, including wearable sensor-based Human Activity Recognition (HAR). In such scenarios, often sensor data are directly fed into an LLM along with text instructions for the model to perform activity classification. Seemingly remarkable results have been reported for such LLM-based HAR systems when they are evaluated on standard benchmarks from the field. Yet, we argue, care has to be taken when evaluating LLM-based HAR systems in such a traditional way. Most contemporary LLMs are trained on virtually the entire (accessible) internet -- potentially including standard HAR datasets. With that, it is not unlikely that LLMs actually had access to the test data used in such benchmark experiments.The resulting contamination of training data would render these experimental evaluations meaningless. In this paper we investigate whether LLMs indeed have had access to standard HAR datasets during training. We apply memorization tests to LLMs, which involves instructing the models to extend given snippets of data. When comparing the LLM-generated output to the original data we found a non-negligible amount of matches which suggests that the LLM under investigation seems to indeed have seen wearable sensor data from the benchmark datasets during training. For the Daphnet dataset in particular, GPT-4 is able to reproduce blocks of sensor readings. We report on our investigations and discuss potential implications on HAR research, especially with regards to reporting results on experimental evaluation