mlsanityData to 14 Oct 2026

Instruct-Video2Avatar: Video-to-Avatar Generation with Instructions

arXiv 2306.02903 ↗First posted 5 Jun 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

We propose a method for synthesizing edited photo-realistic digital avatars with text instructions. Given a short monocular RGB video and text instructions, our method uses an image-conditioned diffusion model to edit one head image and uses the video stylization method to accomplish the editing of other head images. Through iterative training and update (three times or more), our method synthesizes edited photo-realistic animatable 3D neural head avatars with a deformable neural radiance field head synthesis method. In quantitative and qualitative studies on various subjects, our method outperforms state-of-the-art methods.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 43 of its 48 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

43 of 48 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  6. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  7. 2022Video Diffusion Modelscited by 2,305
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  9. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  10. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  11. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  12. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
31 more
  1. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  2. 2015A Neural Algorithm of Artistic Stylecited by 483
  3. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  4. 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
  5. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 370
  6. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  7. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
  8. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 231
  9. 2021Editing Conditional Radiance Fieldscited by 222
  10. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 217
  11. 2021Neural Head Avatars from Monocular RGB Videoscited by 213
  12. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 178
  13. 2019Style Transfer by Relaxed Optimal Transport and Self-Similaritycited by 168
  14. 2022Instant Volumetric Head Avatarscited by 159
  15. 2022StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learningcited by 140
  16. 2022NeRF-Art: Text-Driven Neural Radiance Fields Stylizationcited by 126
  17. 2021Stylizing 3D Scene via Implicit Representation and HyperNetworkcited by 103
  18. 2016Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesiscited by 102
  19. 2017Multi-Content GAN for Few-Shot Font Style Transfercited by 100
  20. 2017Coherent Online Video Style Transfercited by 96
  21. 2022Deforming Radiance Fields with Cagescited by 78
  22. 2021StyleMesh: Style Transfer for Indoor 3D Scene Reconstructionscited by 58
  23. 2016Artistic style transfer for videoscited by 55
  24. 2019Attention-aware Multi-stroke Style Transfercited by 47
  25. 2022DATID-3D: Diversity-Preserved Domain Adaptation Using Text-to-Image Diffusion for 3D Generative Modelcited by 32
  26. 2020Deformable Style Transfercited by 31
  27. 2016Awesome Typography: Statistics-Based Text Effects Transfercited by 24
  28. 2021Exemplar-Based 3D Portrait Stylizationcited by 21
  29. 2023DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 20
  30. 2019Face-to-Parameter Translation for Game Character Auto-Creationcited by 14
  31. 2023Interactive Geometry Editing of Neural Radiance Fieldscited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4; 2025: 1.

6 papers.

  1. 2023A Survey on Video Diffusion Modelscited by 206
  2. 2024InstructAvatar: Text-Guided Emotion and Motion Control for Avatar Generationcited by 16
  3. 2024PEGASUS: Personalized Generative 3D Avatars with Composable Attributescited by 7
  4. 2024LeGO: Leveraging a Surface Deformation Network for Animatable Stylized Face Generation with One Examplecited by 4
  5. 2024AniArtAvatar: Animatable 3D Art Avatar from a Single Imagecited by 2
  6. 2025StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars
  2. 2026Edit3DGS: Unified Framework for Dynamic Head Editing via 2D Instruction-Guided Diffusion and 3D Gaussian Splatting
  3. 2024AvatarMMC: 3D Head Avatar Generation and Editing with Multi-Modal Conditioning
  4. 2023Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Models
  5. 2024DynamicAvatars: Accurate Dynamic Facial Avatars Reconstruction and Precise Editing with Diffusion Models
  6. 2024Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation
  7. 2024HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting
  8. 2025Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video Diffusion