# Game of LLMs: Discovering Structural Constructs in Activities using Large Language Models

arXiv: https://arxiv.org/abs/2406.13777 · HTML: https://mlsanity.com/paper/2406.13777/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 56 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 13 of 13)

- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [VirtualHome: Simulating Household Activities via Programs](https://mlsanity.com/paper/1806.07011/) (2018)
- [Ensembles of Deep LSTM Learners for Activity Recognition using Wearables](https://mlsanity.com/paper/1703.09370/) (2017)
- [SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data](https://mlsanity.com/paper/2102.06073/) (2021)
- [Improving Deep Learning for HAR with shallow LSTMs](https://mlsanity.com/paper/2108.00702/) (2021)
- [Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data](https://mlsanity.com/paper/2206.02353/) (2022)
- [A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning](https://mlsanity.com/paper/2111.04418/) (2021)
- [From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape](https://mlsanity.com/paper/2312.10868/) (2023)
- [Sasha: Creative Goal-Oriented Reasoning in Smart Homes with Large Language Models](https://mlsanity.com/paper/2305.09802/) (2023)
- [Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition](https://mlsanity.com/paper/2306.01108/) (2023)
- [Augmenting Bag-of-Words: Data-Driven Discovery of Temporal and Structural Information for Activity Recognition](https://mlsanity.com/paper/1510.02071/) (2015)
- [Beyond Isolated Frames: Enhancing Sensor-Based Human Activity Recognition through Intra- and Inter-Frame Attention](https://mlsanity.com/paper/2405.19349/) (2024)

## Built on by (most cited first, 1 of 1)

- [AgentSense: Virtual Sensor Data Generation Using LLM Agents in Simulated Home Environments](https://mlsanity.com/paper/2506.11773/) (2025)

## Abstract (arXiv)

Human Activity Recognition is a time-series analysis problem. A popular analysis procedure used by the community assumes an optimal window length to design recognition pipelines. However, in the scenario of smart homes, where activities are of varying duration and frequency, the assumption of a constant sized window does not hold. Additionally, previous works have shown these activities to be made up of building blocks. We focus on identifying these underlying building blocks--structural constructs, with the use of large language models. Identifying these constructs can be beneficial especially in recognizing short-duration and infrequent activities. We also propose the development of an activity recognition procedure that uses these building blocks to model activities, thus helping the downstream task of activity monitoring in smart homes.