# Temporally Consistent Semantic Video Editing

arXiv: https://arxiv.org/abs/2206.10590 · HTML: https://mlsanity.com/paper/2206.10590/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 64 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 30 of 52)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Learning Blind Video Temporal Consistency](https://mlsanity.com/paper/1808.00449/) (2018)
- [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)

## Built on by (most cited first, 28 of 28)

- [Pix2Video: Video Editing using Image Diffusion](https://mlsanity.com/paper/2303.12688/) (2023)
- [StableVideo: Text-driven Consistency-aware Diffusion Video Editing](https://mlsanity.com/paper/2308.09592/) (2023)
- [Shape-aware Text-driven Layered Video Editing](https://mlsanity.com/paper/2301.13173/) (2023)
- [Diffusion Video Autoencoders: Toward Temporally Consistent Face Video Editing via Disentangled Video Encoding](https://mlsanity.com/paper/2212.02802/) (2022)
- [Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generation](https://mlsanity.com/paper/2211.12824/) (2022)
- [Blind Video Deflickering by Neural Filtering with a Flawed Atlas](https://mlsanity.com/paper/2303.08120/) (2023)
- [MEGANE: Morphable Eyeglass and Avatar Network](https://mlsanity.com/paper/2302.04868/) (2023)
- [VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs](https://mlsanity.com/paper/2303.15893/) (2023)
- [Soundini: Sound-Guided Diffusion for Natural Video Editing](https://mlsanity.com/paper/2304.06818/) (2023)
- [RIGID: Recurrent GAN Inversion and Editing of Real Face Videos](https://mlsanity.com/paper/2308.06097/) (2023)
- [INVE: Interactive Neural Video Editing](https://mlsanity.com/paper/2307.07663/) (2023)
- [Contrastive Decoupled Representation Learning and Regularization for Speech-Preserving Facial Expression Manipulation](https://mlsanity.com/paper/2504.05672/) (2025)
- [Text-driven Visual Synthesis with Latent Diffusion Prior](https://mlsanity.com/paper/2302.08510/) (2023)
- [DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs](https://mlsanity.com/paper/2302.12253/) (2023)
- [Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields](https://mlsanity.com/paper/2307.12909/) (2023)
- [Hashing Neural Video Decomposition with Multiplicative Residuals in Space-Time](https://mlsanity.com/paper/2309.14022/) (2023)
- [FLAIR: A Conditional Diffusion Framework with Applications to Face Video Restoration](https://mlsanity.com/paper/2311.15445/) (2023)
- [In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing](https://mlsanity.com/paper/2302.04871/) (2023)
- [Stylized Face Sketch Extraction via Generative Prior with Limited Data](https://mlsanity.com/paper/2403.11263/) (2024)
- [Learning Spatial-Temporal Coherent Correlations for Speech-Preserving Facial Expression Manipulation](https://mlsanity.com/paper/2604.20226/) (2026)
- [FED-NeRF: Achieve High 3D Consistency and Temporal Coherence for Face Video Editing on Dynamic NeRF](https://mlsanity.com/paper/2401.02616/) (2024)
- [Analysis and Benchmarking of Extending Blind Face Image Restoration to Videos](https://mlsanity.com/paper/2410.11828/) (2024)
- [We never go out of Style: Motion Disentanglement by Subspace Decomposition of Latent Space](https://mlsanity.com/paper/2306.00559/) (2023)
- [S3Editor: A Sparse Semantic-Disentangled Self-Training Framework for Face Video Editing](https://mlsanity.com/paper/2404.08111/) (2024)
- [SST-EM: Advanced Metrics for Evaluating Semantic, Spatial and Temporal Aspects in Video Editing](https://mlsanity.com/paper/2501.07554/) (2025)
- [FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing](https://mlsanity.com/paper/2505.22141/) (2025)
- [RS-SSM: Refining Forgotten Specifics in State Space Model for Video Semantic Segmentation](https://mlsanity.com/paper/2603.24295/) (2026)
- [Highly Detailed and Temporal Consistent Video Stylization via Synchronized Multi-Frame Diffusion](https://mlsanity.com/paper/2311.14343/) (2023)

## Abstract (arXiv)

Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e.g., changing object classes, modifying attributes, or transferring styles. However, applying these GAN-based editing to a video independently for each frame inevitably results in temporal flickering artifacts. We present a simple yet effective method to facilitate temporally coherent video editing. Our core idea is to minimize the temporal photometric inconsistency by optimizing both the latent code and the pre-trained generator. We evaluate the quality of our editing on different domains and GAN inversion techniques and show favorable results against the baselines.