mlsanityData to 14 Oct 2026

eMotion-GAN: A Motion-based GAN for Photorealistic and Facial Expression Preserving Frontal View Synthesis

arXiv 2404.09940 ↗First posted 15 Apr 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Many existing facial expression recognition (FER) systems encounter substantial performance degradation when faced with variations in head pose. Numerous frontalization methods have been proposed to enhance these systems' performance under such conditions. However, they often introduce undesirable deformations, rendering them less suitable for precise facial expression analysis. In this paper, we present eMotion-GAN, a novel deep learning approach designed for frontal view synthesis while preserving facial expressions within the motion domain. Considering the motion induced by head variation as noise and the motion induced by facial expression as the relevant information, our model is trained to filter out the noisy motion in order to retain only the motion related to facial expression. The filtered motion is then mapped onto a neutral frontal face to generate the corresponding expressive frontal face. We conducted extensive evaluations using several widely recognized dynamic FER datasets, which encompass sequences exhibiting various degrees of head pose variations in both intensity and orientation. Our results demonstrate the effectiveness of our approach in significantly reducing the FER performance gap between frontal and non-frontal faces. Specifically, we achieved a FER improvement of up to +5% for small pose variations and up to +20% improvement for larger pose variations. Code available at urlhttps://github.com/o-ikne/eMotion-GAN.git.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 72 references that are in the map. First posted on arXiv 15 Apr 2024.

Builds on

28 of 72 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  7. 2014Conditional Generative Adversarial Netscited by 2,162
  8. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  9. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  10. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 492
  11. 2018Deep Facial Expression Recognition: A Surveycited by 270
  12. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
16 more
  1. 2017Video Frame Interpolation via Adaptive Separable Convolutioncited by 213
  2. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  3. 2021Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognitioncited by 89
  4. 2017Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesiscited by 75
  5. 2021Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognitioncited by 55
  6. 2017Towards Large-Pose Face Frontalization in the Wildcited by 50
  7. 2020Cascade EF-GAN: Progressive Facial Expression Editing with Local Focusescited by 43
  8. 2021DRB-GAN: A Dynamic ResBlock Generative Adversarial Network for Artistic Style Transfercited by 39
  9. 2020Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Imagescited by 37
  10. 2021Supervision by Registration and Triangulation for Landmark Detectioncited by 30
  11. 2021Learning Facial Representations from the Cycle-consistency of Facecited by 20
  12. 2019Impact of facial landmark localization on facial expression recognitioncited by 7
  13. 2020Dynamic Facial Expression Recognition under Partial Occlusion with Optical Flow Reconstructioncited by 7
  14. 2019Optical Flow Techniques for Facial Expression Analysis -- a Practical Evaluation Studycited by 6
  15. 2020DeepFaceFlow: In-the-wild Dense 3D Facial Motion Estimationcited by 4
  16. 2022Expression-preserving face frontalization improves visually assisted speech processingcited by 4

Built on by

Not cited yet.