# Perceptually Validated Precise Local Editing for Facial Action Units with StyleGAN

arXiv: https://arxiv.org/abs/2107.12143 · HTML: https://mlsanity.com/paper/2107.12143/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 27 of 27)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [X2Face: A network for controlling face generation by using images, audio, and pose codes](https://mlsanity.com/paper/1807.10550/) (2018)
- [Editing in Style: Uncovering the Local Semantics of GANs](https://mlsanity.com/paper/2004.14367/) (2020)
- [Inverting The Generator Of A Generative Adversarial Network](https://mlsanity.com/paper/1611.05644/) (2016)
- [STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing](https://mlsanity.com/paper/1904.09709/) (2019)
- [Invertible Conditional GANs for image editing](https://mlsanity.com/paper/1611.06355/) (2016)
- [Using latent space regression to analyze and leverage compositionality in GANs](https://mlsanity.com/paper/2103.10426/) (2021)
- [ICface: Interpretable and Controllable Face Reenactment Using GANs](https://mlsanity.com/paper/1904.01909/) (2019)
- [ExprGAN: Facial Expression Editing with Controllable Expression Intensity](https://mlsanity.com/paper/1709.03842/) (2017)
- [Cascade EF-GAN: Progressive Facial Expression Editing with Local Focuses](https://mlsanity.com/paper/2003.05905/) (2020)
- [Geometry Guided Adversarial Facial Expression Synthesis](https://mlsanity.com/paper/1712.03474/) (2017)
- [Spatially Controllable Image Synthesis with Internal Representation Collaging](https://mlsanity.com/paper/1811.10153/) (2018)
- [Geometry-Contrastive GAN for Facial Expression Transfer](https://mlsanity.com/paper/1802.01822/) (2018)
- [Deep Identity-aware Transfer of Facial Attributes](https://mlsanity.com/paper/1610.05586/) (2016)
- [Photorealistic Facial Expression Synthesis by the Conditional Difference Adversarial Autoencoder](https://mlsanity.com/paper/1708.09126/) (2017)
- [Face Identity Disentanglement via Latent Space Mapping](https://mlsanity.com/paper/2005.07728/) (2020)
- [Face Transfer with Generative Adversarial Network](https://mlsanity.com/paper/1710.06090/) (2017)

## Built on by (most cited first, 1 of 1)

- [CLIP-PAE: Projection-Augmentation Embedding to Extract Relevant Features for a Disentangled, Interpretable, and Controllable Text-Guided Face Manipulation](https://mlsanity.com/paper/2210.03919/) (2022)

## Abstract (arXiv)

The ability to edit facial expressions has a wide range of applications in computer graphics. The ideal facial expression editing algorithm needs to satisfy two important criteria. First, it should allow precise and targeted editing of individual facial actions. Second, it should generate high fidelity outputs without artifacts. We build a solution based on StyleGAN, which has been used extensively for semantic manipulation of faces. As we do so, we add to our understanding of how various semantic attributes are encoded in StyleGAN. In particular, we show that a naive strategy to perform editing in the latent space results in undesired coupling between certain action units, even if they are conceptually distinct. For example, although brow lowerer and lip tightener are distinct action units, they appear correlated in the training data. Hence, StyleGAN has difficulty in disentangling them. We allow disentangled editing of such action units by computing detached regions of influence for each action unit, and restrict editing to these regions. We validate the effectiveness of our local editing method through perception experiments conducted with 23 subjects. The results show that our method provides higher control over local editing and produces images with superior fidelity compared to the state-of-the-art methods.