# A Latent Transformer for Disentangled Face Editing in Images and Videos

arXiv: https://arxiv.org/abs/2106.11895 · HTML: https://mlsanity.com/paper/2106.11895/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 46 references that are in the map.
- First posted on arXiv 22 Jun 2021.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [GANalyze: Toward Visual Definitions of Cognitive Image Properties](https://mlsanity.com/paper/1906.10112/) (2019)
- [Editing in Style: Uncovering the Local Semantics of GANs](https://mlsanity.com/paper/2004.14367/) (2020)
- [Adversarial Latent Autoencoders](https://mlsanity.com/paper/2004.04467/) (2020)
- [STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing](https://mlsanity.com/paper/1904.09709/) (2019)
- [Fader Networks: Manipulating Images by Sliding Attributes](https://mlsanity.com/paper/1706.00409/) (2017)
- [Attribute2Image: Conditional Image Generation from Visual Attributes](https://mlsanity.com/paper/1512.00570/) (2015)
- [Deep Feature Interpolation for Image Content Changes](https://mlsanity.com/paper/1611.05507/) (2016)

## Built on by (most cited first, 30 of 67)

- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [Stitch it in Time: GAN-Based Facial Editing of Real Videos](https://mlsanity.com/paper/2201.08361/) (2022)
- [Fine-Grained Face Swapping via Regional GAN Inversion](https://mlsanity.com/paper/2211.14068/) (2022)
- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [Third Time's the Charm? Image and Video Editing with StyleGAN3](https://mlsanity.com/paper/2201.13433/) (2022)
- [Diffusion Video Autoencoders: Toward Temporally Consistent Face Video Editing via Disentangled Video Encoding](https://mlsanity.com/paper/2212.02802/) (2022)
- [Style Transformer for Image Inversion and Editing](https://mlsanity.com/paper/2203.07932/) (2022)
- [Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model](https://mlsanity.com/paper/2111.13333/) (2021)
- [State-of-the-Art in the Architecture, Methods and Applications of StyleGAN](https://mlsanity.com/paper/2202.14020/) (2022)
- [StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesis](https://mlsanity.com/paper/2203.15799/) (2022)
- [Temporally Consistent Semantic Video Editing](https://mlsanity.com/paper/2206.10590/) (2022)
- [User-Controllable Latent Transformer for StyleGAN Image Layout Editing](https://mlsanity.com/paper/2208.12408/) (2022)
- [Finding Directions in GAN's Latent Space for Neural Face Reenactment](https://mlsanity.com/paper/2202.00046/) (2022)
- [MEGANE: Morphable Eyeglass and Avatar Network](https://mlsanity.com/paper/2302.04868/) (2023)
- [VToonify: Controllable High-Resolution Portrait Video Style Transfer](https://mlsanity.com/paper/2209.11224/) (2022)
- [Custom Structure Preservation in Face Aging](https://mlsanity.com/paper/2207.11025/) (2022)
- [Feature-Style Encoder for Style-Based GAN Inversion](https://mlsanity.com/paper/2202.02183/) (2022)
- [GAN-based Facial Attribute Manipulation](https://mlsanity.com/paper/2210.12683/) (2022)
- [VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs](https://mlsanity.com/paper/2303.15893/) (2023)
- [RIGID: Recurrent GAN Inversion and Editing of Real Face Videos](https://mlsanity.com/paper/2308.06097/) (2023)
- [Text-to-image Editing by Image Information Removal](https://mlsanity.com/paper/2305.17489/) (2023)
- [GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning](https://mlsanity.com/paper/2402.02003/) (2024)
- [Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis](https://mlsanity.com/paper/2207.10257/) (2022)
- [MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detection](https://mlsanity.com/paper/2409.09724/) (2024)
- [Adaptive Nonlinear Latent Transformation for Conditional Face Editing](https://mlsanity.com/paper/2307.07790/) (2023)
- [IA-FaceS: A Bidirectional Method for Semantic Face Editing](https://mlsanity.com/paper/2203.13097/) (2022)
- [E4S: Fine-grained Face Swapping via Editing With Regional GAN Inversion](https://mlsanity.com/paper/2310.15081/) (2023)
- [Self-Conditioned Generative Adversarial Networks for Image Editing](https://mlsanity.com/paper/2202.04040/) (2022)
- [One-shot Neural Face Reenactment via Finding Directions in GAN's Latent Space](https://mlsanity.com/paper/2402.03553/) (2024)
- [Multimodal Fake News Detection: MFND Dataset and Shallow-Deep Multitask Learning](https://mlsanity.com/paper/2505.06796/) (2025)

## Abstract (arXiv)

High quality facial image editing is a challenging problem in the movie post-production industry, requiring a high degree of control and identity preservation. Previous works that attempt to tackle this problem may suffer from the entanglement of facial attributes and the loss of the person's identity. Furthermore, many algorithms are limited to a certain task. To tackle these limitations, we propose to edit facial attributes via the latent space of a StyleGAN generator, by training a dedicated latent transformation network and incorporating explicit disentanglement and identity preservation terms in the loss function. We further introduce a pipeline to generalize our face editing to videos. Our model achieves a disentangled, controllable, and identity-preserving facial attribute editing, even in the challenging case of real (i.e., non-synthetic) images and videos. We conduct extensive experiments on image and video datasets and show that our model outperforms other state-of-the-art methods in visual quality and quantitative evaluation. Source codes are available at https://github.com/InterDigitalInc/latent-transformer.