# GesSure- A Robust Face-Authentication enabled Dynamic Gesture Recognition GUI Application

arXiv: https://arxiv.org/abs/2207.11033 · HTML: https://mlsanity.com/paper/2207.11033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 42 references that are in the map.
- First posted on arXiv 22 Jul 2022.

## Builds on (most cited first, 7 of 7)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Domain Adaptation for sEMG-based Gesture Recognition with Recurrent Neural Networks](https://mlsanity.com/paper/1901.06958/) (2019)
- [Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines](https://mlsanity.com/paper/1604.03225/) (2016)
- [Learning Deep and Compact Models for Gesture Recognition](https://mlsanity.com/paper/1712.10136/) (2017)
- [Hybrid Approach to Face Recognition System using Principle component and Independent component with score based fusion process](https://mlsanity.com/paper/1401.0395/) (2014)

## Abstract (arXiv)

Using physical interactive devices like mouse and keyboards hinders naturalistic human-machine interaction and increases the probability of surface contact during a pandemic. Existing gesture-recognition systems do not possess user authentication, making them unreliable. Static gestures in current gesture-recognition technology introduce long adaptation periods and reduce user compatibility. Our technology places a strong emphasis on user recognition and safety. We use meaningful and relevant gestures for task operation, resulting in a better user experience. This paper aims to design a robust, face-verification-enabled gesture recognition system that utilizes a graphical user interface and primarily focuses on security through user recognition and authorization. The face model uses MTCNN and FaceNet to verify the user, and our LSTM-CNN architecture for gesture recognition, achieving an accuracy of 95% with five classes of gestures. The prototype developed through our research has successfully executed context-dependent tasks like save, print, control video-player operations and exit, and context-free operating system tasks like sleep, shut-down, and unlock intuitively. Our application and dataset are available as open source.