mlsanityData to 7 Oct 2026

3D Cartoon Face Generation with Controllable Expressions from a Single GAN Image

arXiv 2207.14425 ↗First posted 29 Jul 2022cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate an open research task of generating 3D cartoon face shapes from single 2D GAN generated human faces and without 3D supervision, where we can also manipulate the facial expressions of the 3D shapes. To this end, we discover the semantic meanings of StyleGAN latent space, such that we are able to produce face images of various expressions, poses, and lighting conditions by controlling the latent codes. Specifically, we first finetune the pretrained StyleGAN face model on the cartoon datasets. By feeding the same latent codes to face and cartoon generation models, we aim to realize the translation from 2D human face images to cartoon styled avatars. We then discover semantic directions of the GAN latent space, in an attempt to change the facial expressions while preserving the original identity. As we do not have any 3D annotations for cartoon faces, we manipulate the latent codes to generate images with different poses and lighting conditions, such that we can reconstruct the 3D cartoon face shapes. We validate the efficacy of our method on three cartoon datasets qualitatively and quantitatively.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 52 references that are in the map. First posted on arXiv 29 Jul 2022.

Builds on

42 of 52 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  6. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  7. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  8. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  10. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  11. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
30 more
  1. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  2. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  3. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  4. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  5. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  6. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  7. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  8. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  9. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  10. 2017DualGAN: Unsupervised Dual Learning for Image-to-Image Translationcited by 310
  11. 2017Learning to Discover Cross-Domain Relations with Generative Adversarial Networkscited by 278
  12. 2019SEAN: Image Synthesis with Semantic Region-Adaptive Normalizationcited by 253
  13. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  14. 2019GANalyze: Toward Visual Definitions of Cognitive Image Propertiescited by 193
  15. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  16. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  17. 2020Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANscited by 132
  18. 2016Unsupervised Cross-Domain Image Generationcited by 125
  19. 2021Image-to-Image Translation: Methods and Applicationscited by 116
  20. 2019Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesiscited by 106
  21. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  22. 2020Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domainscited by 96
  23. 2017Semantic Image Synthesis via Adversarial Learningcited by 87
  24. 2018Cross-View Image Synthesis using Conditional GANscited by 69
  25. 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64
  26. 2020Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANscited by 63
  27. 2018Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Predictioncited by 52
  28. 2021Cycle-Consistent Inverse GAN for Text-to-Image Synthesiscited by 27
  29. 2021Fine-Tuning StyleGAN2 For Cartoon Face Generationcited by 12
  30. 2021Unified Resource Allocation Framework for the Edge Intelligence-Enabled Metaversecited by 6

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 1.

2 papers.

  1. 2022Face Generation and Editing with StyleGAN: A Surveycited by 27
  2. 2024ToonAging: Face Re-Aging upon Artistic Portrait Style Transfercited by 2