# MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech

arXiv: https://arxiv.org/abs/2204.00145 · HTML: https://mlsanity.com/paper/2204.00145/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 144 references that are in the map.
- First posted on arXiv 1 Apr 2022.

## Builds on (most cited first, 10 of 10)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [The Microsoft 2017 Conversational Speech Recognition System](https://mlsanity.com/paper/1708.06073/) (2017)
- [The State of Speech in HCI: Trends, Themes and Challenges](https://mlsanity.com/paper/1810.06828/) (2018)
- [Meta-HAR: Federated Representation Learning for Human Activity Recognition](https://mlsanity.com/paper/2106.00615/) (2021)
- [InChorus: Designing Consistent Multimodal Interactions for Data Visualization on Tablet Devices](https://mlsanity.com/paper/2001.06423/) (2020)
- [Comparing Speech and Keyboard Text Entry for Short Messages in Two Languages on Touchscreen Phones](https://mlsanity.com/paper/1608.07323/) (2016)
- [Crowdsourcing the Perception of Machine Teaching](https://mlsanity.com/paper/2002.01618/) (2020)
- [Exploring Machine Teaching with Children](https://mlsanity.com/paper/2109.11434/) (2021)
- [Refining Automatic Speech Recognition System for older adults](https://mlsanity.com/paper/2011.08346/) (2020)
- [Feasibility of Using Automatic Speech Recognition with Voices of Deaf and Hard-of-Hearing Individuals](https://mlsanity.com/paper/1909.01167/) (2019)

## Built on by (most cited first, 5 of 5)

- [Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data](https://mlsanity.com/paper/2301.05843/) (2023)
- [PlanFitting: Personalized Exercise Planning with Large Language Model-driven Conversational Agent](https://mlsanity.com/paper/2309.12555/) (2023)
- [Leveraging Pre-Trained Language Models to Streamline Natural Language Interaction for Self-Tracking](https://mlsanity.com/paper/2205.15503/) (2022)
- [FARPLS: A Feature-Augmented Robot Trajectory Preference Labeling System to Assist Human Labelers' Preference Elicitation](https://mlsanity.com/paper/2403.06267/) (2024)
- [Characterizing the Performance Gap in Human Activity Recognition for Older Adults](https://mlsanity.com/paper/2610.02711/) (2026)

## Abstract (arXiv)

Current activity tracking technologies are largely trained on younger adults' data, which can lead to solutions that are not well-suited for older adults. To build activity trackers for older adults, it is crucial to collect training data with them. To this end, we examine the feasibility and challenges with older adults in collecting activity labels by leveraging speech. Specifically, we built MyMove, a speech-based smartwatch app to facilitate the in-situ labeling with a low capture burden. We conducted a 7-day deployment study, where 13 older adults collected their activity labels and smartwatch sensor data, while wearing a thigh-worn activity monitor. Participants were highly engaged, capturing 1,224 verbal reports in total. We extracted 1,885 activities with corresponding effort level and timespan, and examined the usefulness of these reports as activity labels. We discuss the implications of our approach and the collected dataset in supporting older adults through personalized activity tracking technologies.