# Learn an Effective Lip Reading Model without Pains

arXiv: https://arxiv.org/abs/2011.07557 · HTML: https://mlsanity.com/paper/2011.07557/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2020.

## Built on by (most cited first, 19 of 19)

- [Training Strategies for Improved Lip-reading](https://mlsanity.com/paper/2209.01383/) (2022)
- [Deep Learning for Visual Speech Analysis: A Survey](https://mlsanity.com/paper/2205.10839/) (2022)
- [Lip Reading for Low-resource Languages by Learning and Combining General Speech Knowledge and Language-specific Knowledge](https://mlsanity.com/paper/2308.09311/) (2023)
- [SyncVSR: Data-Efficient Visual Speech Recognition with End-to-End Crossmodal Audio Token Synchronization](https://mlsanity.com/paper/2406.12233/) (2024)
- [LRWR: Large-Scale Benchmark for Lip Reading in Russian language](https://mlsanity.com/paper/2109.06692/) (2021)
- [LipLearner: Customizable Silent Speech Interactions on Mobile Devices](https://mlsanity.com/paper/2302.05907/) (2023)
- [Advances and Challenges in Deep Lip Reading](https://mlsanity.com/paper/2110.07879/) (2021)
- [Personalized One-Shot Lipreading for an ALS Patient](https://mlsanity.com/paper/2111.01740/) (2021)
- [LipGen: Viseme-Guided Lip Video Generation for Enhancing Visual Speech Recognition](https://mlsanity.com/paper/2501.04204/) (2025)
- [RN-Net: Reservoir Nodes-Enabled Neuromorphic Vision Sensing Network](https://mlsanity.com/paper/2303.10770/) (2023)
- [MTGA: Multi-View Temporal Granularity Aligned Aggregation for Event-Based Lip-Reading](https://mlsanity.com/paper/2404.11979/) (2024)
- [Learning Separable Hidden Unit Contributions for Speaker-Adaptive Lip-Reading](https://mlsanity.com/paper/2310.05058/) (2023)
- [SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer](https://mlsanity.com/paper/2505.04394/) (2025)
- [Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading](https://mlsanity.com/paper/2108.03543/) (2021)
- [Lip-Listening: Mixing Senses to Understand Lips using Cross Modality Knowledge Distillation for Word-Based Models](https://mlsanity.com/paper/2207.05692/) (2022)
- [End-to-End Lip Reading in Romanian with Cross-Lingual Domain Adaptation and Lateral Inhibition](https://mlsanity.com/paper/2310.04858/) (2023)
- [Perception Point: Identifying Critical Learning Periods in Speech for Bilingual Networks](https://mlsanity.com/paper/2110.06507/) (2021)
- [TD3Net: A temporal densely connected multi-dilated convolutional network for lipreading](https://mlsanity.com/paper/2506.16073/) (2025)
- [InfoSyncNet: Information Synchronization Temporal Convolutional Network for Visual Speech Recognition](https://mlsanity.com/paper/2508.02460/) (2025)

## Abstract (arXiv)

Lip reading, also known as visual speech recognition, aims to recognize the speech content from videos by analyzing the lip dynamics. There have been several appealing progress in recent years, benefiting much from the rapidly developed deep learning techniques and the recent large-scale lip-reading datasets. Most existing methods obtained high performance by constructing a complex neural network, together with several customized training strategies which were always given in a very brief description or even shown only in the source code. We find that making proper use of these strategies could always bring exciting improvements without changing much of the model. Considering the non-negligible effects of these strategies and the existing tough status to train an effective lip reading model, we perform a comprehensive quantitative study and comparative analysis, for the first time, to show the effects of several different choices for lip reading. By only introducing some easy-to-get refinements to the baseline pipeline, we obtain an obvious improvement of the performance from 83.7% to 88.4% and from 38.2% to 55.7% on two largest public available lip reading datasets, LRW and LRW-1000, respectively. They are comparable and even surpass the existing state-of-the-art results.