mlsanityData to 14 Oct 2026

Controllable Radiance Fields for Dynamic Face Synthesis

arXiv 2210.05825 ↗First posted 11 Oct 2022cs.CV cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent work on 3D-aware image synthesis has achieved compelling results using advances in neural rendering. However, 3D-aware synthesis of face dynamics hasn't received much attention. Here, we study how to explicitly control generative model synthesis of face dynamics exhibiting non-rigid motion (e.g., facial expression change), while simultaneously ensuring 3D-awareness. For this we propose a Controllable Radiance Field (CoRF): 1) Motion control is achieved by embedding motion features within the layered latent motion space of a style-based generator; 2) To ensure consistency of background, motion features and subject-specific attributes such as lighting, texture, shapes, albedo, and identity, a face parsing net, a head regressor and an identity encoder are incorporated. On head image/video data we show that CoRFs are 3D-aware while enabling editing of identity, viewing directions, and motion.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 62 of its 77 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

62 of 77 references.

References · 62

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  3. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  6. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  7. 2017Improved Training of Wasserstein GANscited by 2,235
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  10. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  11. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  12. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
38 more
  1. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  2. 2016Least Squares Generative Adversarial Networkscited by 1,001
  3. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  4. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  5. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  6. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  7. 2018Self-Attention Generative Adversarial Networkscited by 809
  8. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  9. 2020First Order Motion Model for Image Animationcited by 704
  10. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  11. 2018Which Training Methods for GANs do actually Converge?cited by 679
  12. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  13. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  14. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 632
  15. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  16. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  17. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 525
  18. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  19. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  20. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  21. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  22. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 444
  23. 2016Generating Videos with Scene Dynamicscited by 376
  24. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 342
  25. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
  26. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  27. 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
  28. 2018Deep Video Portraitscited by 267
  29. 2021Motion Representations for Articulated Animationcited by 255
  30. 2018The relativistic discriminator: a key element missing from standard GANcited by 238
  31. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  32. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 203
  33. 2016Temporal Generative Adversarial Nets with Singular Value Clippingcited by 198
  34. 2019Few-shot Video-to-Video Synthesiscited by 197
  35. 2018X2Face: A network for controlling face generation by using images, audio, and pose codescited by 189
  36. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  37. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  38. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 1; 2025: 2; 2026: 1.

8 papers.

  1. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  2. 2024Generalizable and Animatable Gaussian Head Avatarcited by 97
  3. 2025LAM: Large Avatar Model for One-shot Animatable Gaussian Headcited by 43
  4. 2022PV3D: A 3D Generative Model for Portrait Video Generationcited by 23
  5. 2023Generalizable One-shot Neural Head Avatarcited by 21
  6. 2023From Pixels to Portraits: A Comprehensive Survey of Talking Head Generation Techniques and Applicationscited by 9
  7. 2025Total-Editing: Head Avatar with Editable Appearance, Motion, and Lightingcited by 2
  8. 2026MeshLAM: Feed-Forward One-Shot Animatable Textured Mesh Avatar Reconstructioncited by 1