# FacEnhance: Facial Expression Enhancing with Recurrent DDPMs

arXiv: https://arxiv.org/abs/2406.09040 · HTML: https://mlsanity.com/paper/2406.09040/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 31 of its 41 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 30 of 31)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [Generating Videos with Scene Dynamics](https://mlsanity.com/paper/1609.02612/) (2016)
- [Animating Arbitrary Objects via Deep Motion Transfer](https://mlsanity.com/paper/1812.08861/) (2018)
- [X2Face: A network for controlling face generation by using images, audio, and pose codes](https://mlsanity.com/paper/1807.10550/) (2018)
- [Conditional Image-to-Video Generation with Latent Flow Diffusion Models](https://mlsanity.com/paper/2303.13744/) (2023)
- [DCFace: Synthetic Face Generation with Dual Condition Diffusion Model](https://mlsanity.com/paper/2304.07060/) (2023)
- [Neural Head Reenactment with Latent Pose Descriptors](https://mlsanity.com/paper/2004.12000/) (2020)
- [Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching](https://mlsanity.com/paper/1911.04131/) (2019)
- [S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation](https://mlsanity.com/paper/2005.11437/) (2020)
- [ExprGAN: Facial Expression Editing with Controllable Expression Intensity](https://mlsanity.com/paper/1709.03842/) (2017)
- [StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis](https://mlsanity.com/paper/2203.15799/) (2022)
- [Geometry Guided Adversarial Facial Expression Synthesis](https://mlsanity.com/paper/1712.03474/) (2017)
- [A Survey of Diffusion Based Image Generation Models: Issues and Their Solutions](https://mlsanity.com/paper/2308.13142/) (2023)
- [Dynamic Facial Expression Generation on Hilbert Hypersphere with Conditional Wasserstein Generative Adversarial Nets](https://mlsanity.com/paper/1907.10087/) (2019)
- [Geometry-Contrastive GAN for Facial Expression Transfer](https://mlsanity.com/paper/1802.01822/) (2018)
- [TwoStreamVAN: Improving Motion Modeling in Video Generation](https://mlsanity.com/paper/1812.01037/) (2018)
- [Toward Fine-grained Facial Expression Manipulation](https://mlsanity.com/paper/2004.03132/) (2020)
- [Language-Guided Face Animation by Recurrent StyleGAN-based Generator](https://mlsanity.com/paper/2208.05617/) (2022)
- [Unconstrained Facial Expression Transfer using Style-based Generator](https://mlsanity.com/paper/1912.06253/) (2019)
- [Joint Deep Learning of Facial Expression Synthesis and Recognition](https://mlsanity.com/paper/2002.02194/) (2020)
- [US-GAN: On the importance of Ultimate Skip Connection for Facial Expression Synthesis](https://mlsanity.com/paper/2112.13002/) (2021)
- [Facial Expression Analysis Using Decomposed Multiscale Spatiotemporal Networks](https://mlsanity.com/paper/2203.11111/) (2022)
- [Facial Expression Video Generation Based-On Spatio-temporal Convolutional GAN: FEV-GAN](https://mlsanity.com/paper/2210.11182/) (2022)

## Built on by (most cited first, 1 of 1)

- [ARGen: Affect-Reinforced Generative Augmentation towards Vision-based Dynamic Emotion Perception](https://mlsanity.com/paper/2604.12255/) (2026)

## Abstract (arXiv)

Facial expressions, vital in non-verbal human communication, have found applications in various computer vision fields like virtual reality, gaming, and emotional AI assistants. Despite advancements, many facial expression generation models encounter challenges such as low resolution (e.g., 32x32 or 64x64 pixels), poor quality, and the absence of background details. In this paper, we introduce FacEnhance, a novel diffusion-based approach addressing constraints in existing low-resolution facial expression generation models. FacEnhance enhances low-resolution facial expression videos (64x64 pixels) to higher resolutions (192x192 pixels), incorporating background details and improving overall quality. Leveraging conditional denoising within a diffusion framework, guided by a background-free low-resolution video and a single neutral expression high-resolution image, FacEnhance generates a video incorporating the facial expression from the low-resolution video performed by the individual with background from the neutral image. By complementing lightweight low-resolution models, FacEnhance strikes a balance between computational efficiency and desirable image resolution and quality. Extensive experiments on the MUG facial expression database demonstrate the efficacy of FacEnhance in enhancing low-resolution model outputs to state-of-the-art quality while preserving content and identity consistency. FacEnhance represents significant progress towards resource-efficient, high-fidelity facial expression generation, Renewing outdated low-resolution methods to up-to-date standards.