# Modular Deep Learning Framework for Assistive Perception: Gaze, Affect, and Speaker Identification

arXiv: https://arxiv.org/abs/2511.20474 · HTML: https://mlsanity.com/paper/2511.20474/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 28 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 2 of 2)

- [Facial Expression Recognition from World Wild Web](https://mlsanity.com/paper/1605.03639/) (2016)
- [A Survey on Eye-Gaze Tracking Techniques](https://mlsanity.com/paper/1312.6410/) (2013)

## Abstract (arXiv)

Developing comprehensive assistive technologies requires the seamless integration of visual and auditory perception. This research evaluates the feasibility of a modular architecture inspired by core functionalities of perceptive systems like 'Smart Eye.' We propose and benchmark three independent sensing modules: a Convolutional Neural Network (CNN) for eye state detection (drowsiness/attention), a deep CNN for facial expression recognition, and a Long Short-Term Memory (LSTM) network for voice-based speaker identification. Utilizing the Eyes Image, FER2013, and customized audio datasets, our models achieved accuracies of 93.0%, 97.8%, and 96.89%, respectively. This study demonstrates that lightweight, domain-specific models can achieve high fidelity on discrete tasks, establishing a validated foundation for future real-time, multimodal integration in resource-constrained assistive devices.