mlsanityData to 7 Oct 2026

GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image

arXiv 2404.02152 ↗First posted 2 Apr 2024cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Recently, we have witnessed the explosive growth of various volumetric representations in modeling animatable head avatars. However, due to the diversity of frameworks, there is no practical method to support high-level applications like 3D head avatar editing across different representations. In this paper, we propose a generic avatar editing approach that can be universally applied to various 3DMM driving volumetric head avatars. To achieve this goal, we design a novel expression-aware modification generative model, which enables lift 2D editing from a single image to a consistent 3D modification field. To ensure the effectiveness of the generative modification process, we develop several techniques, including an expression-dependent modification distillation scheme to draw knowledge from the large-scale head avatar model and 2D facial texture editing tools, implicit latent space guidance to enhance model convergence, and a segmentation-based loss reweight strategy for fine-grained texture inversion. Extensive experiments demonstrate that our method delivers high-quality and consistent results across multiple expression and viewpoints. Project page: https://zju3dv.github.io/geneavatar/

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 60 of its 74 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

60 of 74 references.

References · 60

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  4. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  5. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  6. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  8. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  9. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  10. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  11. 2020Neural Sparse Voxel Fieldscited by 959
  12. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
38 more
  1. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  2. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  3. 2023Zero-shot Image-to-Image Translationcited by 503
  4. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  5. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  6. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  7. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  8. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  9. 2021GAN Inversion: A Surveycited by 366
  10. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  11. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  12. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  13. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  14. 2021Learning Object-Compositional Neural Radiance Field for Editable Scene Renderingcited by 228
  15. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  16. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  17. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 183
  18. 2022Towards Metrical Reconstruction of Human Facescited by 165
  19. 2022Instant Volumetric Head Avatarscited by 157
  20. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  21. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  22. 2022Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representationcited by 149
  23. 2022NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editingcited by 141
  24. 2020DeepFaceLab: Integrated, flexible and extensible face-swapping frameworkcited by 129
  25. 2022Reconstructing Personalized Semantic Facial NeRF Models From Monocular Videocited by 128
  26. 2022RigNeRF: Fully Controllable Neural 3D Portraitscited by 126
  27. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  28. 20213D-aware Image Synthesis via Learning Structural and Textural Representationscited by 109
  29. 2023SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Fieldcited by 93
  30. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  31. 2020Learning Compositional Radiance Fields of Dynamic Human Headscited by 78
  32. 2022AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxelscited by 73
  33. 2023SKED: Sketch-guided Text-based 3D Editingcited by 72
  34. 2019PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfercited by 66
  35. 2022IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesiscited by 50
  36. 2020Navigating the GAN Parameter Space for Semantic Image Editingcited by 49
  37. 2022AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatarscited by 46
  38. 2023Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videoscited by 44

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 3.

12 papers.