Representation Learning of Daily Movement Data Using Text Encoders
Not on the map. Why?
Abstract (arXiv)
Time-series representation learning is a key area of research for remote healthcare monitoring applications. In this work, we focus on a dataset of recordings of in-home activity from people living with Dementia. We design a representation learning method based on converting activity to text strings that can be encoded using a language model fine-tuned to transform data from the same participants within a 30-day window to similar embeddings in the vector space. This allows for clustering and vector searching over participants and days, and the identification of activity deviations to aid with personalised delivery of care.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 36 references that are in the map. First posted on arXiv 7 May 2024.
Builds on
22 of 36 references.
References · 22
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
- 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
- 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
10 more
- 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
- 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
- 2014Deep metric learning using Triplet networkcited by 421
- 2021A Survey of Transformerscited by 399
- 2022Multimodal Learning with Transformers: A Surveycited by 399
- 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 342
- 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 267
- 2023A Cookbook of Self-Supervised Learningcited by 248
- 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
- 2022Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusioncited by 96
Built on by
Citing papers per year: 2025: 2.
2 papers.