mlsanityData to 7 Oct 2026

MUNCH: Modelling Unique 'N Controllable Heads

arXiv 2310.02753 ↗First posted 4 Oct 2023cs.CV cs.AI cs.GRcited by 0

Not on the map. Why?

Abstract (arXiv)

The automated generation of 3D human heads has been an intriguing and challenging task for computer vision researchers. Prevailing methods synthesize realistic avatars but with limited control over the diversity and quality of rendered outputs and suffer from limited correlation between shape and texture of the character. We propose a method that offers quality, diversity, control, and realism along with explainable network design, all desirable features to game-design artists in the domain. First, our proposed Geometry Generator identifies disentangled latent directions and generate novel and diverse samples. A Render Map Generator then learns to synthesize multiply high-fidelty physically-based render maps including Albedo, Glossiness, Specular, and Normals. For artists preferring fine-grained control over the output, we introduce a novel Color Transformer Model that allows semantic color control over generated maps. We also introduce quantifiable metrics called Uniqueness and Novelty and a combined metric to test the overall performance of our model. Demo for both shapes and textures can be found: https://munch-seven.vercel.app/. We will release our model along with the synthetic dataset.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

31 of 49 references.

References · 31

  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  2. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  4. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  5. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 267
  6. 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 257
  7. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  8. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  9. 2017Morphable Face Models - An Open Frameworkcited by 138
  10. 2020Learning Formation of Physically-Based Face Attributescited by 94
  11. 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
  12. 2019Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generationcited by 89
19 more
  1. 2019SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operatorcited by 89
  2. 2019Towards a complete 3D morphable model of the human headcited by 63
  3. 2017Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translationcited by 56
  4. 2019Combining 3D Morphable Models: A Large scale Face-and-Head Modelcited by 49
  5. 2021AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANscited by 49
  6. 2023FitMe: Deep Photorealistic 3D Morphable Model Avatarscited by 38
  7. 2020OSTeC: One-Shot Texture Completioncited by 37
  8. 2019Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networkscited by 34
  9. 2021Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstructioncited by 33
  10. 2016Photorealistic Facial Texture Inference Using Deep Neural Networkscited by 32
  11. 2022DAD-3DHeads: A Large-scale Dense, Accurate and Diverse Dataset for 3D Head Alignment from a Single Imagecited by 31
  12. 20193DFaceGAN: Adversarial Nets for 3D Face Representation, Generation, and Translationcited by 27
  13. 2018High Quality Facial Surface and Texture Synthesis via Generative Adversarial Networkscited by 22
  14. 2019Dense 3D Face Decoding over 2500FPS: Joint Texture & Shape Convolutional Mesh Decoderscited by 22
  15. 2020A 3D GAN for Improved Large-pose Facial Recognitioncited by 19
  16. 2021MeInGame: Create a Game Character Face from a Single Portraitcited by 16
  17. 2020CA-GAN: Weakly Supervised Color Aware GAN for Controllable Makeup Transfercited by 15
  18. 20213D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Facescited by 15
  19. 2022Controllable 3D Generative Adversarial Face Model via Disentangling Shape and Appearancecited by 8

Built on by

Not cited yet.