mlsanityData to 7 Oct 2026

OPHAvatars: One-shot Photo-realistic Head Avatars

arXiv 2307.09153 ↗First posted 18 Jul 2023cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We propose a method for synthesizing photo-realistic digital avatars from only one portrait as the reference. Given a portrait, our method synthesizes a coarse talking head video using driving keypoints features. And with the coarse video, our method synthesizes a coarse talking head avatar with a deforming neural radiance field. With rendered images of the coarse avatar, our method updates the low-quality images with a blind face restoration model. With updated images, we retrain the avatar for higher quality. After several iterations, our method can synthesize a photo-realistic animatable 3D neural head avatar. The motivation of our method is deformable neural radiance field can eliminate the unnatural distortion caused by the image2video method. Our method outperforms state-of-the-art methods in quantitative and qualitative studies on various subjects.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 42 of its 51 references that are in the map. First posted on arXiv 18 Jul 2023.

Builds on

42 of 51 references.

References · 42

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  4. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  5. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  6. 2020First Order Motion Model for Image Animationcited by 698
  7. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  8. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
  9. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  10. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  11. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  12. 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
30 more
  1. 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
  2. 2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 347
  3. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  4. 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 310
  5. 2020BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentationcited by 294
  6. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  7. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  8. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  9. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  10. 2021GAN Prior Embedded Network for Blind Face Restoration in the Wildcited by 197
  11. 2019Image Processing Using Multi-Code GAN Priorcited by 185
  12. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  13. 2017FSRNet: End-to-End Learning Face Super-Resolution with Facial Priorscited by 170
  14. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  15. 2022Towards Metrical Reconstruction of Human Facescited by 165
  16. 2022Instant Volumetric Head Avatarscited by 157
  17. 2021Robust High-Resolution Video Matting with Temporal Guidancecited by 136
  18. 2020Blind Face Restoration via Deep Multi-scale Component Dictionariescited by 106
  19. 2020HeadGAN: One-shot Neural Head Synthesis and Editingcited by 105
  20. 2020Progressive Semantic-Aware Style Transformation for Blind Face Restorationcited by 99
  21. 2018Learning Warped Guidance for Blind Face Restorationcited by 87
  22. 2022Multiface: A Dataset for Neural Face Renderingcited by 87
  23. 2022RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairscited by 85
  24. 2019Progressive Face Super-Resolution via Attention to Facial Landmarkcited by 73
  25. 2023OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane Renderingcited by 67
  26. 2019Exemplar Guided Face Image Super-Resolution without Facial Landmarkscited by 57
  27. 2020Learning Spatial Attention for Face Super-Resolutioncited by 55
  28. 2016Deep Cascaded Bi-Network for Face Hallucinationcited by 37
  29. 2022Deep Face Restoration: A Surveycited by 17
  30. 2022Blind Face Restoration: Benchmark Datasets and a Baseline Modelcited by 10

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesiscited by 97