# Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual Drivers

arXiv: https://arxiv.org/abs/2401.16123 · HTML: https://mlsanity.com/paper/2401.16123/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 61 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Builds on (most cited first, 14 of 14)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions](https://mlsanity.com/paper/2204.08817/) (2022)
- [Process Outcome Prediction: CNN vs. LSTM (with Attention)](https://mlsanity.com/paper/2104.06934/) (2021)
- [You Have a Point There: Object Selection Inside an Automobile Using Gaze, Head Pose and Finger Pointing](https://mlsanity.com/paper/2012.13449/) (2020)
- [ML-PersRef: A Machine Learning-based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle](https://mlsanity.com/paper/2111.02327/) (2021)
- [Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle](https://mlsanity.com/paper/2009.11195/) (2020)
- [Multimodal Fusion Using Deep Learning Applied to Driver's Referencing of Outside-Vehicle Objects](https://mlsanity.com/paper/2107.12167/) (2021)
- [Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces](https://mlsanity.com/paper/2211.03539/) (2022)
- [It's all about you: Personalized in-Vehicle Gesture Recognition with a Time-of-Flight Camera](https://mlsanity.com/paper/2310.01659/) (2023)

## Built on by (most cited first, 1 of 1)

- [Speak to the City: Multimodal Resolution for Outside-the-Vehicle References](https://mlsanity.com/paper/2609.14691/) (2026)

## Abstract (arXiv)

The rapid advancement of the automotive industry towards automated and semi-automated vehicles has rendered traditional methods of vehicle interaction, such as touch-based and voice command systems, inadequate for a widening range of non-driving related tasks, such as referencing objects outside of the vehicle. Consequently, research has shifted toward gestural input (e.g., hand, gaze, and head pose gestures) as a more suitable mode of interaction during driving. However, due to the dynamic nature of driving and individual variation, there are significant differences in drivers' gestural input performance. While, in theory, this inherent variability could be moderated by substantial data-driven machine learning models, prevalent methodologies lean towards constrained, single-instance trained models for object referencing. These models show a limited capacity to continuously adapt to the divergent behaviors of individual drivers and the variety of driving scenarios. To address this, we propose textitIcRegress, a novel regression-based incremental learning approach that adapts to changing behavior and the unique characteristics of drivers engaged in the dual task of driving and referencing objects. We suggest a more personalized and adaptable solution for multimodal gestural interfaces, employing continuous lifelong learning to enhance driver experience, safety, and convenience. Our approach was evaluated using an outside-the-vehicle object referencing use case, highlighting the superiority of the incremental learning models adapted over a single trained model across various driver traits such as handedness, driving experience, and numerous driving conditions. Finally, to facilitate reproducibility, ease deployment, and promote further research, we offer our approach as an open-source framework at urlhttps://github.com/amrgomaaelhady/IcRegress.