# Deep Learning-based Spatio Temporal Facial Feature Visual Speech Recognition

arXiv: https://arxiv.org/abs/2305.00552 · HTML: https://mlsanity.com/paper/2305.00552/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 30 Apr 2023.

## Builds on (most cited first, 12 of 12)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [Deep Learning Face Representation by Joint Identification-Verification](https://mlsanity.com/paper/1406.4773/) (2014)
- [Large-Scale Visual Speech Recognition](https://mlsanity.com/paper/1807.05162/) (2018)
- [DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection](https://mlsanity.com/paper/1412.5661/) (2014)
- [Targeting Ultimate Accuracy: Face Recognition via Deep Embedding](https://mlsanity.com/paper/1506.07310/) (2015)
- [Deep learning and face recognition: the state of the art](https://mlsanity.com/paper/1902.03524/) (2019)
- [Video Vision Transformers for Violence Detection](https://mlsanity.com/paper/2209.03561/) (2022)
- [LesionAid: Vision Transformers-based Skin Lesion Generation and Classification](https://mlsanity.com/paper/2302.01104/) (2023)
- [Vision Transformers and YoloV5 based Driver Drowsiness Detection Framework](https://mlsanity.com/paper/2209.01401/) (2022)
- [A Novel end-to-end Framework for Occluded Pixel Reconstruction with Spatio-temporal Features for Improved Person Re-identification](https://mlsanity.com/paper/2304.07721/) (2023)
- [AuthNet: A Deep Learning based Authentication Mechanism using Temporal Facial Feature Movements](https://mlsanity.com/paper/2012.02515/) (2020)

## Abstract (arXiv)

In low-resource computing contexts, such as smartphones and other tiny devices, Both deep learning and machine learning are being used in a lot of identification systems. as authentication techniques. The transparent, contactless, and non-invasive nature of these face recognition technologies driven by AI has led to their meteoric rise in popularity in recent years. While they are mostly successful, there are still methods to get inside without permission by utilising things like pictures, masks, glasses, etc. In this research, we present an alternate authentication process that makes use of both facial recognition and the individual's distinctive temporal facial feature motions while they speak a password. Because the suggested methodology allows for a password to be specified in any language, it is not limited by language. The suggested model attained an accuracy of 96.1% when tested on the industry-standard MIRACL-VC1 dataset, demonstrating its efficacy as a reliable and powerful solution. In addition to being data-efficient, the suggested technique shows promising outcomes with as little as 10 positive video examples for training the model. The effectiveness of the network's training is further proved via comparisons with other combined facial recognition and lip reading models.