mlsanityData to 7 Oct 2026

PEGASUS: Personalized Generative 3D Avatars with Composable Attributes

arXiv 2402.10636 ↗First posted 16 Feb 2024cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We present PEGASUS, a method for constructing a personalized generative 3D face avatar from monocular video sources. Our generative 3D avatar enables disentangled controls to selectively alter the facial attributes (e.g., hair or nose) while preserving the identity. Our approach consists of two stages: synthetic database generation and constructing a personalized generative avatar. We generate a synthetic video collection of the target identity with varying facial attributes, where the videos are synthesized by borrowing the attributes from monocular videos of diverse identities. Then, we build a person-specific generative 3D avatar that can modify its attributes continuously while preserving its identity. Through extensive experiments, we demonstrate that our method of generating a synthetic database and creating a 3D generative avatar is the most effective in preserving identity while achieving high realism. Subsequently, we introduce a zero-shot approach to achieve the same goal of generative modeling more efficiently by leveraging a previously constructed personalized generative model.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 62 of its 83 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

62 of 83 references.

References · 62

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
38 more
  1. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  4. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  5. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  6. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
  7. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  8. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  9. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  10. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  11. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  12. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  13. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  14. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  15. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  16. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 183
  17. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  18. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  19. 2022Towards Metrical Reconstruction of Human Facescited by 165
  20. 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Modelscited by 154
  21. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  22. 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
  23. 2020MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decompositioncited by 127
  24. 2023Vox-E: Text-guided Voxel Editing of 3D Objectscited by 114
  25. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  26. 2022Capturing and Animation of Body and Clothing from Monocular Videocited by 83
  27. 2023DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidancecited by 79
  28. 2022Stitch it in Time: GAN-Based Facial Editing of Real Videoscited by 68
  29. 2022Fine-Grained Face Swapping via Regional GAN Inversioncited by 67
  30. 2022ClipFace: Text-guided Editing of Textured 3D Morphable Modelscited by 59
  31. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  32. 2023AG3D: Learning to Generate 3D Avatars from 2D Image Collectionscited by 55
  33. 2023Learning Locally Editable Virtual Humanscited by 54
  34. 2023LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatarcited by 50
  35. 2023Instruct 3D-to-3D: Text Instruction Guided 3D-to-3D conversioncited by 47
  36. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  37. 2023Neural Haircut: Prior-Guided Strand-Based Hair Reconstructioncited by 43
  38. 2022Neural Strands: Learning Hair Geometry and Appearance from Multi-View Imagescited by 39

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 2.

7 papers.

  1. 2024PERSE: Personalized 3D Generative Avatars from A Single Portraitcited by 15
  2. 2025HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatarscited by 7
  3. 2025SVAD: From Single Image to 3D Avatar via Synthetic Data Generation with Video Diffusion and Data Augmentationcited by 2
  4. 2026PhysHead: Simulation-Ready Gaussian Head Avatarscited by 2
  5. 2025Durian: Dual Reference Image-Guided Portrait Animation with Attribute Transfercited by 1
  6. 2026Vanast: Virtual Try-On with Human Image Animation via Synthetic Triplet Supervisioncited by 1
  7. 2024Pomo3D: 3D-Aware Portrait Accessorizing and Morecited by 0