# Detecting Activities of Daily Living in Egocentric Video to Contextualize Hand Use at Home in Outpatient Neurorehabilitation Settings

arXiv: https://arxiv.org/abs/2412.10846 · HTML: https://mlsanity.com/paper/2412.10846/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 14 Dec 2024.

## Builds on (most cited first, 13 of 13)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Understanding Human Hands in Contact at Internet Scale](https://mlsanity.com/paper/2006.06669/) (2020)
- [The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines](https://mlsanity.com/paper/2005.00343/) (2020)
- [Trear: Transformer-based RGB-D Egocentric Action Recognition](https://mlsanity.com/paper/2101.03904/) (2021)
- [SOS! Self-supervised Learning Over Sets Of Handled Objects In Egocentric Action Recognition](https://mlsanity.com/paper/2204.04796/) (2022)
- [Measuring hand use in the home after cervical spinal cord injury using egocentric video](https://mlsanity.com/paper/2203.16996/) (2022)
- [A Hybrid Framework for Action Recognition in Low-Quality Video Sequences](https://mlsanity.com/paper/1903.04090/) (2019)
- [EgoViT: Pyramid Video Transformer for Egocentric Action Recognition](https://mlsanity.com/paper/2303.08920/) (2023)

## Built on by (most cited first, 2 of 2)

- [EgoLife: Towards Egocentric Life Assistant](https://mlsanity.com/paper/2503.03803/) (2025)
- [MMeViT: Multi-Modal ensemble ViT for Post-Stroke Rehabilitation Action Recognition](https://mlsanity.com/paper/2509.23044/) (2025)

## Abstract (arXiv)

Wearable egocentric cameras and machine learning have the potential to provide clinicians with a more nuanced understanding of patient hand use at home after stroke and spinal cord injury (SCI). However, they require detailed contextual information (i.e., activities and object interactions) to effectively interpret metrics and meaningfully guide therapy planning. We demonstrate that an object-centric approach, focusing on what objects patients interact with rather than how they move, can effectively recognize Activities of Daily Living (ADL) in real-world rehabilitation settings. We evaluated our models on a complex dataset collected in the wild comprising 2261 minutes of egocentric video from 16 participants with impaired hand function. By leveraging pre-trained object detection and hand-object interaction models, our system achieves robust performance across different impairment levels and environments, with our best model achieving a mean weighted F1-score of 0.78 +/- 0.12 and maintaining an F1-score > 0.5 for all participants using leave-one-subject-out cross validation. Through qualitative analysis, we observe that this approach generates clinically interpretable information about functional object use while being robust to patient-specific movement variations, making it particularly suitable for rehabilitation contexts with prevalent upper limb impairment.