# It's all about you: Personalized in-Vehicle Gesture Recognition with a Time-of-Flight Camera

arXiv: https://arxiv.org/abs/2310.01659 · HTML: https://mlsanity.com/paper/2310.01659/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 65 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy](https://mlsanity.com/paper/2002.04087/) (2020)
- [Real-time Hand Gesture Detection and Classification Using Convolutional Neural Networks](https://mlsanity.com/paper/1901.10323/) (2019)
- [You Have a Point There: Object Selection Inside an Automobile Using Gaze, Head Pose and Finger Pointing](https://mlsanity.com/paper/2012.13449/) (2020)
- [ML-PersRef: A Machine Learning-based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle](https://mlsanity.com/paper/2111.02327/) (2021)
- [Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle](https://mlsanity.com/paper/2009.11195/) (2020)
- [Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces](https://mlsanity.com/paper/2211.03539/) (2022)

## Built on by (most cited first, 2 of 2)

- [Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual Drivers](https://mlsanity.com/paper/2401.16123/) (2024)
- [Speak to the City: Multimodal Resolution for Outside-the-Vehicle References](https://mlsanity.com/paper/2609.14691/) (2026)

## Abstract (arXiv)

Despite significant advances in gesture recognition technology, recognizing gestures in a driving environment remains challenging due to limited and costly data and its dynamic, ever-changing nature. In this work, we propose a model-adaptation approach to personalize the training of a CNNLSTM model and improve recognition accuracy while reducing data requirements. Our approach contributes to the field of dynamic hand gesture recognition while driving by providing a more efficient and accurate method that can be customized for individual users, ultimately enhancing the safety and convenience of in-vehicle interactions, as well as driver's experience and system trust. We incorporate hardware enhancement using a time-of-flight camera and algorithmic enhancement through data augmentation, personalized adaptation, and incremental learning techniques. We evaluate the performance of our approach in terms of recognition accuracy, achieving up to 90%, and show the effectiveness of personalized adaptation and incremental learning for a user-centered design.