mlsanityData to 7 Oct 2026

Deep Learning for Visual Speech Analysis: A Survey

arXiv 2205.10839 ↗First posted 22 May 2022cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

Visual speech, referring to the visual domain of speech, has attracted increasing attention due to its wide applications, such as public security, medical treatment, military defense, and film entertainment. As a powerful AI strategy, deep learning techniques have extensively promoted the development of visual speech learning. Over the past five years, numerous deep learning based methods have been proposed to address various problems in this area, especially automatic visual speech recognition and generation. To push forward future research on visual speech, this paper aims to present a comprehensive review of recent progress in deep learning methods on visual speech analysis. We cover different aspects of visual speech, including fundamental problems, challenges, benchmark datasets, a taxonomy of existing methods, and state-of-the-art performance. Besides, we also identify gaps in current research and discuss inspiring future research directions.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 119 of its 203 references that are in the map. First posted on arXiv 22 May 2022.

Builds on

119 of 203 references.

References · 119

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  8. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  9. 2014Conditional Generative Adversarial Netscited by 2,152
  10. 2020Knowledge Distillation: A Surveycited by 1,578
  11. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  12. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
38 more
  1. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  2. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  3. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  4. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  5. 2020First Order Motion Model for Image Animationcited by 698
  6. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
  7. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  8. 2020A Survey on Visual Transformercited by 653
  9. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  10. 2015Listen, Attend and Spellcited by 542
  11. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  12. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  13. 2015Attention-Based Models for Speech Recognitioncited by 400
  14. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 396
  15. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  16. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  17. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  18. 2018Deep Audio-Visual Speech Recognitioncited by 318
  19. 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 310
  20. 2018LRS3-TED: a large-scale dataset for visual speech recognitioncited by 301
  21. 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
  22. 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
  23. 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
  24. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  25. 2018Deep Video Portraitscited by 265
  26. 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 257
  27. 2017Objects that Soundcited by 254
  28. 2019Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Losscited by 252
  29. 2019Neural Voice Puppetry: Audio-driven Facial Reenactmentcited by 249
  30. 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 246
  31. 2016Lip Reading Sentences in the Wildcited by 226
  32. 2018Talking Face Generation by Adversarially Disentangled Audio-Visual Representationcited by 212
  33. 2018DeepFakes: a New Threat to Face Recognition? Assessment and Detectioncited by 208
  34. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 208
  35. 2021MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglementcited by 189
  36. 2018Learning to Localize Sound Source in Visual Scenescited by 188
  37. 2021Audio-Driven Emotional Video Portraitscited by 180
  38. 2018Lip Movements Generation at a Glancecited by 154

50 most cited of 119 All 119 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 6; 2026: 2.

21 papers.

Sort:

By area:

  1. 2023EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animationcited by 141
  2. 2024Recent Advances in Speech Language Models: A Surveycited by 85
  3. 2023DiffusionTalker: Personalization and Acceleration for Speech-Driven 3D Face Diffusercited by 16
  4. 2024Learn2Talk: 3D Talking Face Learns from 2D Talking Facecited by 16
  5. 2023Lip2Vec: Efficient and Robust Visual Speech Recognition via Latent-to-Latent Visual to Audio Representation Mappingcited by 10
  6. 2023DiffDub: Person-generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-encodercited by 10
  7. 2023A Survey on Deep Multi-modal Learning for Body Language Recognition and Generationcited by 3
  8. 2024MTGA: Multi-View Temporal Granularity Aligned Aggregation for Event-Based Lip-Readingcited by 3
  9. 2024A Multi-Stream Fusion Approach with One-Class Learning for Audio-Visual Deepfake Detectioncited by 3
  10. 2025CBW: Towards Dataset Ownership Verification for Speaker Verification via Clustering-based Backdoor Watermarkingcited by 3
  11. 2024ViSpeR: Multilingual Audio-Visual Speech Recognitioncited by 2
  12. 2024Cross-Modal Few-Shot Learning: a Generative Transfer Learning Frameworkcited by 2
9 more
  1. 2024Cross-Attention Fusion of Visual and Geometric Features for Large Vocabulary Arabic Lipreadingcited by 1
  2. 2025Transfer Learning from Visual Speech Recognition to Mouthing Recognition in German Sign Languagecited by 1
  3. 2025AD-AVSR: Asymmetric Dual-stream Enhancement for Robust Audio-Visual Speech Recognitioncited by 1
  4. 2026Learning Physiology-Informed Vocal Spectrotemporal Representations for Speech Emotion Recognitioncited by 1
  5. 2024Robust Multi-Modal Speech In-Painting: A Sequence-to-Sequence Approachcited by 0
  6. 2025OXSeg: Multidimensional attention UNet-based lip segmentation using semi-supervised lip contourscited by 0
  7. 2025TD3Net: A temporal densely connected multi-dilated convolutional network for lipreadingcited by 0
  8. 2025Toward Efficient Speech Emotion Recognition via Spectral Learning and Attentioncited by 0
  9. 2026AVSRBench: A Multi-Condition AVSR Benchmarkcited by 0