# Automated Speaker Independent Visual Speech Recognition: A Comprehensive Survey

arXiv: https://arxiv.org/abs/2306.08314 · HTML: https://mlsanity.com/paper/2306.08314/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 182 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 22 of 22)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [LRS3-TED: a large-scale dataset for visual speech recognition](https://mlsanity.com/paper/1809.00496/) (2018)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [A Critical Review of Recurrent Neural Networks for Sequence Learning](https://mlsanity.com/paper/1506.00019/) (2015)
- [Visual Speech Recognition for Multiple Languages in the Wild](https://mlsanity.com/paper/2202.13084/) (2022)
- [Generative Adversarial Networks (GANs Survey): Challenges, Solutions, and Future Directions](https://mlsanity.com/paper/2005.00065/) (2020)
- [Challenges and Opportunities in Edge Computing](https://mlsanity.com/paper/1609.01967/) (2016)
- [A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading](https://mlsanity.com/paper/1908.04917/) (2019)
- [Training Strategies for Improved Lip-reading](https://mlsanity.com/paper/2209.01383/) (2022)
- [Spatiotemporal CNN for Video Object Segmentation](https://mlsanity.com/paper/1904.02363/) (2019)
- [Face Mask Detection using Transfer Learning of InceptionV3](https://mlsanity.com/paper/2009.08369/) (2020)
- [Prompt Tuning of Deep Neural Networks for Speaker-adaptive Visual Speech Recognition](https://mlsanity.com/paper/2302.08102/) (2023)
- [LipFormer: Learning to Lipread Unseen Speakers based on Visual-Landmark Transformers](https://mlsanity.com/paper/2302.02141/) (2023)
- [A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learning](https://mlsanity.com/paper/2202.13403/) (2022)
- [Decoding visemes: improving machine lipreading](https://mlsanity.com/paper/1710.01288/) (2017)
- [Resolution limits on visual speech recognition](https://mlsanity.com/paper/1710.01073/) (2017)
- [Is Lip Region-of-Interest Sufficient for Lipreading?](https://mlsanity.com/paper/2205.14295/) (2022)
- [Lip Reading Using Convolutional Auto Encoders as Feature Extractor](https://mlsanity.com/paper/1805.12371/) (2018)
- [OLKAVS: An Open Large-Scale Korean Audio-Visual Speech Dataset](https://mlsanity.com/paper/2301.06375/) (2023)
- [Lip Localization and Viseme Classification for Visual Speech Recognition](https://mlsanity.com/paper/1301.4558/) (2013)

## Abstract (arXiv)

Speaker-independent VSR is a complex task that involves identifying spoken words or phrases from video recordings of a speaker's facial movements. Over the years, there has been a considerable amount of research in the field of VSR involving different algorithms and datasets to evaluate system performance. These efforts have resulted in significant progress in developing effective VSR models, creating new opportunities for further research in this area. This survey provides a detailed examination of the progression of VSR over the past three decades, with a particular emphasis on the transition from speaker-dependent to speaker-independent systems. We also provide a comprehensive overview of the various datasets used in VSR research and the preprocessing techniques employed to achieve speaker independence. The survey covers the works published from 1990 to 2023, thoroughly analyzing each work and comparing them on various parameters. This survey provides an in-depth analysis of speaker-independent VSR systems evolution from 1990 to 2023. It outlines the development of VSR systems over time and highlights the need to develop end-to-end pipelines for speaker-independent VSR. The pictorial representation offers a clear and concise overview of the techniques used in speaker-independent VSR, thereby aiding in the comprehension and analysis of the various methodologies. The survey also highlights the strengths and limitations of each technique and provides insights into developing novel approaches for analyzing visual speech cues. Overall, This comprehensive review provides insights into the current state-of-the-art speaker-independent VSR and highlights potential areas for future research.