# Toward Fine-grained Facial Expression Manipulation

arXiv: https://arxiv.org/abs/2004.03132 · HTML: https://mlsanity.com/paper/2004.03132/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 9 of 9)

- [StableFace: Analyzing and Improving Motion Stability for Talking Face Generation](https://mlsanity.com/paper/2208.13717/) (2022)
- [POCE: Pose-Controllable Expression Editing](https://mlsanity.com/paper/2304.08938/) (2023)
- [PoseTalk: Text-and-Audio-based Pose Control and Motion Refinement for One-Shot Talking Head Generation](https://mlsanity.com/paper/2409.02657/) (2024)
- [A comprehensive survey on semantic facial attribute editing using generative adversarial networks](https://mlsanity.com/paper/2205.10587/) (2022)
- [SARGAN: Spatial Attention-based Residuals for Facial Expression Manipulation](https://mlsanity.com/paper/2303.17212/) (2023)
- [Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities](https://mlsanity.com/paper/2502.06803/) (2025)
- [GaFET: Learning Geometry-aware Facial Expression Translation from In-The-Wild Images](https://mlsanity.com/paper/2308.03413/) (2023)
- [AUEditNet: Dual-Branch Facial Action Unit Intensity Manipulation with Implicit Disentanglement](https://mlsanity.com/paper/2404.05063/) (2024)
- [FacEnhance: Facial Expression Enhancing with Recurrent DDPMs](https://mlsanity.com/paper/2406.09040/) (2024)

## Abstract (arXiv)

Facial expression manipulation aims at editing facial expression with a given condition. Previous methods edit an input image under the guidance of a discrete emotion label or absolute condition (e.g., facial action units) to possess the desired expression. However, these methods either suffer from changing condition-irrelevant regions or are inefficient for fine-grained editing. In this study, we take these two objectives into consideration and propose a novel method. First, we replace continuous absolute condition with relative condition, specifically, relative action units. With relative action units, the generator learns to only transform regions of interest which are specified by non-zero-valued relative AUs. Second, our generator is built on U-Net but strengthened by Multi-Scale Feature Fusion (MSF) mechanism for high-quality expression editing purposes. Extensive experiments on both quantitative and qualitative evaluation demonstrate the improvements of our proposed approach compared to the state-of-the-art expression editing methods. Code is available at urlhttps://github.com/junleen/Expression-manipulator.