mlsanityData to 7 Oct 2026

StyleRig: Rigging StyleGAN for 3D Control over Portrait Images

arXiv 2004.00121 ↗First posted 31 Mar 2020cs.CV cs.GRcited by 267

Not on the map. Why?

Abstract (arXiv)

StyleGAN generates photorealistic portrait images of faces with eyes, teeth, hair and context (neck, shoulders, background), but lacks a rig-like control over semantic face parameters that are interpretable in 3D, such as face pose, expressions, and scene illumination. Three-dimensional morphable face models (3DMMs) on the other hand offer control over the semantic parameters, but lack photorealism when rendered and only model the face interior, not other parts of a portrait image (hair, mouth interior, background). We present the first method to provide a face rig-like control over a pretrained and fixed StyleGAN via a 3DMM. A new rigging network, RigNet is trained between the 3DMM's semantic parameters and StyleGAN's input. The network is trained in a self-supervised manner, without the need for manual annotations. At test time, our method generates portrait images with the photorealism of StyleGAN and provides explicit control over the 3D semantic parameters of the face.

Key facts. Cited by 267 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 65; 2022: 85; 2023: 53; 2024: 36; 2025: 19; 2026: 9.

267 papers.

Sort:

By area:

  1. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  2. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  3. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  4. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  5. 2021GAN Inversion: A Surveycited by 366
  6. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  7. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  8. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  9. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  10. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  11. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  12. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
38 more
  1. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  2. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
  3. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  4. 2022StyleGAN-Human: A Data-Centric Odyssey of Human Generationcited by 143
  5. 2022RigNeRF: Fully Controllable Neural 3D Portraitscited by 126
  6. 2023Relightable Gaussian Codec Avatarscited by 126
  7. 2022Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatarscited by 124
  8. 2021FENeRF: Face Editing in Neural Radiance Fieldscited by 112
  9. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  10. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  11. 2021One Shot Face Swapping on Megapixelscited by 105
  12. 2021GAN-Control: Explicitly Controllable GANscited by 91
  13. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  14. 2023DiffusionRig: Learning Personalized Priors for Facial Appearance Editingcited by 74
  15. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  16. 2023IDiff-Face: Synthetic-based Face Recognition through Fizzy Identity-Conditioned Diffusion Modelscited by 72
  17. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  18. 2021Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networkscited by 68
  19. 2022Stitch it in Time: GAN-Based Facial Editing of Real Videoscited by 68
  20. 2023One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Fieldcited by 68
  21. 2021A Latent Transformer for Disentangled Face Editing in Images and Videoscited by 67
  22. 2021StylePeople: A Generative Model of Fullbody Human Avatarscited by 62
  23. 2023Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Modelscited by 60
  24. 2022StyleLight: HDR Panorama Generation for Lighting Estimation and Editingcited by 59
  25. 2022ClipFace: Text-guided Editing of Textured 3D Morphable Modelscited by 59
  26. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  27. 2024Exploiting Style Latent Flows for Generalizing Deepfake Video Detectioncited by 58
  28. 2021Using latent space regression to analyze and leverage compositionality in GANscited by 55
  29. 2022Third Time's the Charm? Image and Video Editing with StyleGAN3cited by 54
  30. 2021Style and Pose Control for Image Synthesis of Humans from a Single Monocular Viewcited by 53
  31. 2021Labels4Free: Unsupervised Segmentation using StyleGANcited by 53
  32. 20223D GAN Inversion for Controllable Portrait Image Animationcited by 53
  33. 2023StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Videocited by 52
  34. 2025Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatarscited by 52
  35. 2021StyleAlign: Analysis and Applications of Aligned StyleGAN Modelscited by 48
  36. 2023Face Animation with an Attribute-Guided Diffusion Modelcited by 48
  37. 2023HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Facescited by 48
  38. 2021HumanGAN: A Generative Model of Humans Imagescited by 47

50 most cited of 267 All 267 (JSON)