mlsanityData to 14 Oct 2026

Attention-Based Lip Audio-Visual Synthesis for Talking Face Generation in the Wild

arXiv 2203.03984 ↗First posted 8 Mar 2022cs.CV cs.MM eess.IVcited by 7

Not on the map. Why?

Abstract (arXiv)

Talking face generation with great practical significance has attracted more attention in recent audio-visual studies. How to achieve accurate lip synchronization is a long-standing challenge to be further investigated. Motivated by xxx, in this paper, an AttnWav2Lip model is proposed by incorporating spatial attention module and channel attention module into lip-syncing strategy. Rather than focusing on the unimportant regions of the face image, the proposed AttnWav2Lip model is able to pay more attention on the lip region reconstruction. To our limited knowledge, this is the first attempt to introduce attention mechanism to the scheme of talking face generation. An extensive experiments have been conducted to evaluate the effectiveness of the proposed model. Compared to the baseline measured by LSE-D and LSE-C metrics, a superior performance has been demonstrated on the benchmark lip synthesis datasets, including LRW, LRS2 and LRS3.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 23 of its 36 references that are in the map. First posted on arXiv 8 Mar 2022.

Builds on

23 of 36 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  6. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  7. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  8. 2014Conditional Generative Adversarial Netscited by 2,162
  9. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  10. 2015Spatial Transformer Networkscited by 1,429
  11. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  12. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
11 more
  1. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
  2. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  3. 2014Recurrent Models of Visual Attentioncited by 349
  4. 2018Deep Audio-Visual Speech Recognitioncited by 322
  5. 2018LRS3-TED: a large-scale dataset for visual speech recognitioncited by 308
  6. 2018Talking Face Generation by Adversarially Disentangled Audio-Visual Representationcited by 212
  7. 2017S3FD: Single Shot Scale-invariant Face Detectorcited by 142
  8. 2017You said that?cited by 99
  9. 2020Towards Automatic Face-to-Face Translationcited by 96
  10. 2017ObamaNet: Photo-realistic lip-sync from textcited by 42
  11. 2015ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairscited by 35

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 0; 2025: 1; 2026: 1.

7 papers.

  1. 2022MARLIN: Masked Autoencoder for facial video Representation LearnINgcited by 57
  2. 2023Emotionally Enhanced Talking Face Generationcited by 14
  3. 2022An Audio-Visual Attention Based Multimodal Network for Fake Talking Face Videos Detectioncited by 2
  4. 2023HDTR-Net: A Real-Time High-Definition Teeth Restoration Network for Arbitrary Talking Face Generation Methodscited by 2
  5. 2023FTFDNet: Learning to Detect Talking Face Video Manipulation with Tri-Modality Interactioncited by 1
  6. 2025Lightweight Joint Audio-Visual Deepfake Detection via Single-Stream Multi-Modal Learning Frameworkcited by 1
  7. 2026LPIPS-AttnWav2Lip: Generic Audio-Driven lip synchronization for Talking Head Generation in the Wildcited by 0