# Lip-reading with Densely Connected Temporal Convolutional Networks

arXiv: https://arxiv.org/abs/2009.14233 · HTML: https://mlsanity.com/paper/2009.14233/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2020.

## Built on by (most cited first, 26 of 26)

- [LiRA: Learning Visual Speech Representations from Audio through Self-supervision](https://mlsanity.com/paper/2106.09171/) (2021)
- [Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Reading](https://mlsanity.com/paper/2204.01725/) (2022)
- [Learning to Dub Movies via Hierarchical Prosody Models](https://mlsanity.com/paper/2212.04054/) (2022)
- [Training Strategies for Improved Lip-reading](https://mlsanity.com/paper/2209.01383/) (2022)
- [Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network](https://mlsanity.com/paper/2104.10122/) (2021)
- [SyncVSR: Data-Efficient Visual Speech Recognition with End-to-End Crossmodal Audio Token Synchronization](https://mlsanity.com/paper/2406.12233/) (2024)
- [Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks](https://mlsanity.com/paper/2201.02143/) (2022)
- [IIANet: An Intra- and Inter-Modality Attention Network for Audio-Visual Speech Separation](https://mlsanity.com/paper/2308.08143/) (2023)
- [Multi-Temporal Lip-Audio Memory for Visual Speech Recognition](https://mlsanity.com/paper/2305.04542/) (2023)
- [AlignDiT: Multimodal Aligned Diffusion Transformer for Synchronized Speech Generation](https://mlsanity.com/paper/2504.20629/) (2025)
- [CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition Challenge](https://mlsanity.com/paper/2406.10313/) (2024)
- [Advances and Challenges in Deep Lip Reading](https://mlsanity.com/paper/2110.07879/) (2021)
- [Expression-preserving face frontalization improves visually assisted speech processing](https://mlsanity.com/paper/2204.02810/) (2022)
- [A General Model for Detecting Learner Engagement: Implementation and Evaluation](https://mlsanity.com/paper/2405.04251/) (2024)
- [LipGen: Viseme-Guided Lip Video Generation for Enhancing Visual Speech Recognition](https://mlsanity.com/paper/2501.04204/) (2025)
- [The Multimodal Information Based Speech Processing (MISP) 2025 Challenge: Audio-Visual Diarization and Recognition](https://mlsanity.com/paper/2505.13971/) (2025)
- [Towards MOOCs for Lipreading: Using Synthetic Talking Heads to Train Humans in Lipreading at Scale](https://mlsanity.com/paper/2208.09796/) (2022)
- [SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer](https://mlsanity.com/paper/2505.04394/) (2025)
- [Efficient Audio-Visual Speech Separation with Discrete Lip Semantics and Multi-Scale Global-Local Attention](https://mlsanity.com/paper/2509.23610/) (2025)
- [Do Smart Glasses Dream of Sentimental Visions? Deep Emotionship Analysis for Eyewear Devices](https://mlsanity.com/paper/2201.09933/) (2022)
- [RAL:Redundancy-Aware Lipreading Model Based on Differential Learning with Symmetric Views](https://mlsanity.com/paper/2409.05307/) (2024)
- [TD3Net: A temporal densely connected multi-dilated convolutional network for lipreading](https://mlsanity.com/paper/2506.16073/) (2025)
- [InfoSyncNet: Information Synchronization Temporal Convolutional Network for Visual Speech Recognition](https://mlsanity.com/paper/2508.02460/) (2025)
- [Landmark Guided Visual Feature Extractor for Visual Speech Recognition with Limited Resource](https://mlsanity.com/paper/2508.07233/) (2025)
- [Designing Practical Models for Isolated Word Visual Speech Recognition](https://mlsanity.com/paper/2508.17894/) (2025)
- [Cascade-Free Mandarin Visual Speech Recognition via Semantic-Guided Cross-Representation Alignment](https://mlsanity.com/paper/2603.21808/) (2026)

## Abstract (arXiv)

In this work, we present the Densely Connected Temporal Convolutional Network (DC-TCN) for lip-reading of isolated words. Although Temporal Convolutional Networks (TCN) have recently demonstrated great potential in many vision tasks, its receptive fields are not dense enough to model the complex temporal dynamics in lip-reading scenarios. To address this problem, we introduce dense connections into the network to capture more robust temporal features. Moreover, our approach utilises the Squeeze-and-Excitation block, a light-weight attention mechanism, to further enhance the model's classification power. Without bells and whistles, our DC-TCN method has achieved 88.36% accuracy on the Lip Reading in the Wild (LRW) dataset and 43.65% on the LRW-1000 dataset, which has surpassed all the baseline methods and is the new state-of-the-art on both datasets.