mlsanityData to 7 Oct 2026

Diffusion Video Autoencoders: Toward Temporally Consistent Face Video Editing via Disentangled Video Encoding

arXiv 2212.02802 ↗First posted 6 Dec 2022cs.CV cs.AI cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

Inspired by the impressive performance of recent face image editing methods, several studies have been naturally proposed to extend these methods to the face video editing task. One of the main challenges here is temporal consistency among edited frames, which is still unresolved. To this end, we propose a novel face video editing framework based on diffusion autoencoders that can successfully extract the decomposed features - for the first time as a face video editing model - of identity and motion from a given video. This modeling allows us to edit the video by simply manipulating the temporally invariant feature to the desired direction for the consistency. Another unique strength of our model is that, since our model is based on diffusion models, it can satisfy both reconstruction and edit capabilities at the same time, and is robust to corner cases in wild face videos (e.g. occluded faces) unlike the existing GAN-based methods.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 37 of its 45 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

37 of 45 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  12. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
25 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  2. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  3. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  4. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  5. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  6. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  7. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  8. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  9. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  10. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  11. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  12. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  13. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  14. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  15. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 393
  16. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  17. 2020BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentationcited by 294
  18. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  19. 2018Learning Blind Video Temporal Consistencycited by 247
  20. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  21. 2022Stitch it in Time: GAN-Based Facial Editing of Real Videoscited by 68
  22. 2021A Latent Transformer for Disentangled Face Editing in Images and Videoscited by 67
  23. 2022Third Time's the Charm? Image and Video Editing with StyleGAN3cited by 54
  24. 2022Temporally Consistent Semantic Video Editingcited by 28
  25. 2022Near Perfect GAN Inversioncited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 12; 2025: 12; 2026: 6.

39 papers.

Sort:

By area:

  1. 2023A Survey on Video Diffusion Modelscited by 203
  2. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
  3. 2024Deepfake Generation and Detection: A Benchmark and Surveycited by 66
  4. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  5. 2023DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencodercited by 48
  6. 2024A Survey on Generative AI and LLM for Video Generation, Understanding, and Streamingcited by 37
  7. 2022Face Generation and Editing with StyleGAN: A Surveycited by 27
  8. 2024FinePOSE: Fine-Grained Prompt-Driven 3D Human Pose Estimation via Diffusion Modelscited by 23
  9. 2023Hulk: A Universal Knowledge Translator for Human-Centric Taskscited by 22
  10. 2024DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactmentcited by 14
  11. 2023RIGID: Recurrent GAN Inversion and Editing of Real Face Videoscited by 13
  12. 2024CloudFixer: Test-Time Adaptation for 3D Point Clouds via Diffusion-Guided Geometric Transformationcited by 11
27 more
  1. 2023Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updatescited by 9
  2. 2024ConsistentAvatar: Learning to Diffuse Fully Consistent Talking Head Avatar with Temporal Guidancecited by 7
  3. 2023Text-driven Visual Synthesis with Latent Diffusion Priorcited by 6
  4. 2023Video Face Re-Aging: Toward Temporally Consistent Face Re-Agingcited by 3
  5. 2024Video to Video Generative Adversarial Network for Few-shot Learning Based on Policy Gradientcited by 3
  6. 2025DevFD: Developmental Face Forgery Detection by Learning Shared and Orthogonal LoRA Subspacescited by 3
  7. 2024FED-NeRF: Achieve High 3D Consistency and Temporal Coherence for Face Video Editing on Dynamic NeRFcited by 2
  8. 2024ToonAging: Face Re-Aging upon Artistic Portrait Style Transfercited by 2
  9. 2025Motion Diffusion Autoencoders: Enabling Attribute Manipulation in Human Motion Demonstrated on Karate Techniquescited by 2
  10. 2025Detecting Localized Deepfake Manipulations Using Action Unit-Guided Video Representationscited by 2
  11. 2025ReactDiff: Latent Diffusion for Facial Reaction Generationcited by 2
  12. 2024S3Editor: A Sparse Semantic-Disentangled Self-Training Framework for Face Video Editingcited by 1
  13. 2024High-fidelity and Lip-synced Talking Face Synthesis via Landmark-based Diffusion Modelcited by 1
  14. 2025DiViD: Disentangled Video Diffusion for Static-Dynamic Factorizationcited by 1
  15. 2025Durian: Dual Reference Image-Guided Portrait Animation with Attribute Transfercited by 1
  16. 2025Revisiting Diffusion Autoencoder Training for Image Reconstruction Qualitycited by 0
  17. 2025Automated Learning of Semantic Embedding Representations for Diffusion Modelscited by 0
  18. 2025FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editingcited by 0
  19. 2025Efficient Diffusion-Based 3D Human Pose Estimation with Hierarchical Temporal Pruningcited by 0
  20. 2025Diff-3DCap: Shape Captioning with Diffusion Modelscited by 0
  21. 2025NeRV-Diffusion: Diffuse Implicit Neural Representations for Video Synthesiscited by 0
  22. 2026Exploring Talking Head Models With Adjacent Frame Prior for Speech-Preserving Facial Expression Manipulationcited by 0
  23. 2026Direct Discrepancy Replay: Distribution-Discrepancy Condensation and Manifold-Consistent Replay for Continual Face Forgery Detectioncited by 0
  24. 2026DiffMagicFace: Identity Consistent Facial Editing of Real Videoscited by 0
  25. 2026LatRef-Diff: Latent and Reference-Guided Diffusion for Facial Attribute Editing and Style Manipulationcited by 0
  26. 2026AttDiff-GAN: A Hybrid Diffusion-GAN Framework for Facial Attribute Editingcited by 0
  27. 2026ReAge3D: Re-Aging 3D Faces with View Consistencycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025IP-FaceDiff: Identity-Preserving Facial Video Editing with Diffusion
  2. 2024VividFace: A Diffusion-Based Hybrid Framework for High-Fidelity Video Face Swapping
  3. 2026VFace: A Training-Free Approach for Diffusion-Based Video Face Swapping
  4. 2026DVFace: Spatio-Temporal Dual-Prior Diffusion for Video Face Restoration
  5. 2025VividFace: High-Quality and Efficient One-Step Diffusion For Video Face Enhancement
  6. 2023MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers
  7. 2024Blended Latent Diffusion under Attention Control for Real-World Video Editing
  8. 2024Anchored Diffusion for Video Face Reenactment