# OntoAR-HAR: Ontology-Guided Agentic RAG with Personalized Memory for Human Activity Recognition

arXiv: https://arxiv.org/abs/2610.15292 · HTML: https://mlsanity.com/paper/2610.15292/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation.
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 11 Oct 2026.

## Placed in

- Retrieval-augmented generation / [Agentic RAG, agents and memory](https://mlsanity.com/rag/agentic-rag-agents-and-memory/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 20 of 20)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data](https://mlsanity.com/paper/2206.02909/) (2022)
- [HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?](https://mlsanity.com/paper/2403.02727/) (2024)
- [SensorLM: Learning the Language of Wearable Sensors](https://mlsanity.com/paper/2506.09108/) (2025)
- [SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognition](https://mlsanity.com/paper/2410.10624/) (2024)
- [UniMTS: Unified Pre-training for Motion Time Series](https://mlsanity.com/paper/2410.19818/) (2024)
- [Mantis: Lightweight Foundation Model for Time Series Classification](https://mlsanity.com/paper/2502.15637/) (2025)
- [IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition](https://mlsanity.com/paper/2402.01049/) (2024)
- [ZARA: Training-Free Motion Time-Series Reasoning via Evidence-Grounded LLM Agents](https://mlsanity.com/paper/2508.04038/) (2025)
- [Unleashing the Power of Shared Label Structures for Human Activity Recognition](https://mlsanity.com/paper/2301.03462/) (2023)
- [RAG-HAR: Retrieval Augmented Generation-based Human Activity Recognition](https://mlsanity.com/paper/2512.08984/) (2025)
- [Large Language Model-Guided Semantic Alignment for Human Activity Recognition](https://mlsanity.com/paper/2410.00003/) (2024)

## Abstract (arXiv)

LLMs increasingly support sensor-based human activity recognition (HAR) by linking motion signals with activity semantics and physical knowledge. Retrieval-augmented generation (RAG) grounds decisions in sensor examples and domain features, enabling training-free target-user inference with less dependence on raw numerical arrays. However, existing LLM/RAG-based HAR typically searches single-level activity spaces using static population knowledge. Consequently, they compare many unrelated candidates using the same static population evidence and overlook personalized motion patterns. We propose OntoAR-HAR, an ontology-guided agentic RAG framework combining static population and personalized knowledge. Its tree-structured activity ontology decomposes classification into local decisions among semantically related activities. The static store supplies node-constrained examples, while offline-discovered executable signal features provide numerical evidence for local distinctions. During inference, the personalized store collects selected user observations, which the LLM integrates with static knowledge and signal features while traversing the ontology to an activity label. Cross-subject and heterogeneous-sensing experiments show complementary gains from hierarchical retrieval and personalized memory across multiple LLM backbones, achieving state-of-the-art performance. By explicitly modeling hierarchical relations across activity granularities, OntoAR-HAR also addresses the long-overlooked semantic inclusion problem and avoids treating related labels as independent categories.