# PAL: Intelligence Augmentation using Egocentric Visual Context Detection

arXiv: https://arxiv.org/abs/2105.10735 · HTML: https://mlsanity.com/paper/2105.10735/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 32 references that are in the map.
- First posted on arXiv 22 May 2021.

## Builds on (most cited first, 6 of 6)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Predicting Daily Activities From Egocentric Images Using Deep Learning](https://mlsanity.com/paper/1510.01576/) (2015)
- [PAL: A Wearable Platform for Real-time, Personalized and Context-Aware Health and Cognition Support](https://mlsanity.com/paper/1905.01352/) (2019)

## Built on by (most cited first, 1 of 1)

- [Personalizing Pre-trained Models](https://mlsanity.com/paper/2106.01499/) (2021)

## Abstract (arXiv)

Egocentric visual context detection can support intelligence augmentation applications. We created a wearable system, called PAL, for wearable, personalized, and privacy-preserving egocentric visual context detection. PAL has a wearable device with a camera, heart-rate sensor, on-device deep learning, and audio input/output. PAL also has a mobile/web application for personalized context labeling. We used on-device deep learning models for generic object and face detection, low-shot custom face and context recognition (e.g., activities like brushing teeth), and custom context clustering (e.g., indoor locations). The models had over 80% accuracy in in-the-wild contexts (~1000 images) and we tested PAL for intelligence augmentation applications like behavior change. We have made PAL is open-source to further support intelligence augmentation using personalized and privacy-preserving egocentric visual contexts.