mlsanityData to 7 Oct 2026

Mimic3D: Thriving 3D-Aware GANs via 3D-to-2D Imitation

arXiv 2303.09036 ↗First posted 16 Mar 2023cs.CV cs.GRcited by 18

Not on the map. Why?

Abstract (arXiv)

Generating images with both photorealism and multiview 3D consistency is crucial for 3D-aware GANs, yet existing methods struggle to achieve them simultaneously. Improving the photorealism via CNN-based 2D super-resolution can break the strict 3D consistency, while keeping the 3D consistency by learning high-resolution 3D representations for direct rendering often compromises image quality. In this paper, we propose a novel learning strategy, namely 3D-to-2D imitation, which enables a 3D-aware GAN to generate high-quality images while maintaining their strict 3D consistency, by letting the images synthesized by the generator's 3D rendering branch to mimic those generated by its 2D super-resolution branch. We also introduce 3D-aware convolutions into the generator for better 3D representation learning, which further improves the image generation quality. With the above strategies, our method reaches FID scores of 5.4 and 4.3 on FFHQ and AFHQ-v2 Cats, respectively, at 512x512 resolution, largely outperforming existing 3D-aware GANs using direct 3D rendering and coming very close to the previous state-of-the-art method that leverages 2D super-resolution. Project website: https://seanchenxy.github.io/Mimic3DWeb.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 48 of its 56 references that are in the map. First posted on arXiv 16 Mar 2023.

Builds on

48 of 56 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  11. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  12. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
36 more
  1. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  2. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  4. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  5. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  6. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  9. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  10. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  11. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  12. 2018Which Training Methods for GANs do actually Converge?cited by 672
  13. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  14. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  15. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  16. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  17. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  18. 2019On the "steerability" of generative adversarial networkscited by 247
  19. 2021GRAM: Generative Radiance Manifolds for 3D-Aware Image Generationcited by 216
  20. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  21. 2021CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesiscited by 149
  22. 2022VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Gridscited by 133
  23. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  24. 20213D-aware Image Synthesis via Learning Structural and Textural Representationscited by 109
  25. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  26. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  27. 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
  28. 2022GRAM-HD: 3D-Consistent Image Generation at High Resolution with Generative Radiance Manifoldscited by 73
  29. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  30. 2020Lifting 2D StyleGAN for 3D-Aware Face Generationcited by 64
  31. 2020Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANscited by 63
  32. 2019Unsupervised Generative 3D Shape Learning from Natural Imagescited by 57
  33. 2022Generative Multiplane Images: Making a 2D GAN 3D-Awarecited by 57
  34. 2021Generative Occupancy Fields for 3D Surface-Aware Image Synthesiscited by 51
  35. 2022Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesiscited by 43
  36. 2022GAN2X: Non-Lambertian Inverse Rendering of Image GANscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 8; 2025: 5; 2026: 1.

18 papers.

Sort:

By area:

  1. 2023Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Datacited by 55
  2. 2024HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priorscited by 44
  3. 2024GaussianMarker: Uncertainty-Aware Copyright Protection of 3D Gaussian Splattingcited by 20
  4. 2023CAD: Photorealistic 3D Generation via Adversarial Distillationcited by 18
  5. 2024What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANscited by 12
  6. 2025MarkSplatter: Generalizable Watermarking for 3D Gaussian Splatting Model via Splatter Image Structurecited by 10
  7. 2024GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splatscited by 9
  8. 20233DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Posescited by 8
  9. 20253DGH: 3D Head Generation with Composable Hair and Facecited by 8
  10. 2025PanoLAM: Large Avatar Model for Gaussian Full-Head Synthesis from One-shot Unposed Imagecited by 7
  11. 2024GeometrySticker: Enabling Ownership Claim of Recolorized Neural Radiance Fieldscited by 6
  12. 2025CGS-GAN: 3D Consistent Gaussian Splatting GANs for High Resolution Human Head Synthesiscited by 6
6 more
  1. 2024RaFE: Generative Radiance Fields Restorationcited by 4
  2. 2024Learning 3D-Aware GANs from Unposed Images with Template Feature Fieldcited by 3
  3. 2024Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Priorcited by 2
  4. 2025S3D: Sketch-Driven 3D Model Generationcited by 1
  5. 2026Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generationcited by 1
  6. 2023GVP: Generative Volumetric Primitivescited by 0