mlsanityData to 7 Oct 2026

Learning Generative Models of Textured 3D Meshes from Real-World Images

arXiv 2103.15627 ↗First posted 29 Mar 2021cs.CV cs.GR cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

Recent advances in differentiable rendering have sparked an interest in learning generative models of textured 3D meshes from image collections. These models natively disentangle pose and appearance, enable downstream applications in computer graphics, and improve the ability of generative models to understand the concept of image formation. Although there has been prior work on learning such models from collections of 2D images, these approaches require a delicate pose estimation step that exploits annotated keypoints, thereby restricting their applicability to a few specific datasets. In this work, we propose a GAN framework for generating textured triangle meshes without relying on such annotations. We show that the performance of our approach is on par with prior work that relies on ground-truth keypoints, and more importantly, we demonstrate the generality of our method by setting new baselines on a larger set of categories from ImageNet - for which keypoints are not available - without any class-specific hyperparameter tuning. We release our code at https://github.com/dariopavllo/textured-3d-gan

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 48 of its 58 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

48 of 58 references.

References · 48

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  10. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  11. 2018Self-Attention Generative Adversarial Networkscited by 809
  12. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
36 more
  1. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  2. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  3. 2019PointRend: Image Segmentation as Renderingcited by 439
  4. 2017Neural 3D Mesh Renderercited by 437
  5. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  6. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  7. 2018cGANs with Projection Discriminatorcited by 253
  8. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  9. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  10. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  11. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  12. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  13. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  14. 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
  15. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  16. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  17. 2020Shape and Viewpoint without Keypointscited by 92
  18. 2018Multiresolution Tree Networks for 3D Point Cloud Processingcited by 91
  19. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  20. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  21. 2020Articulation-aware Canonical Surface Mappingcited by 80
  22. 2020Leveraging 2D Data to Learn Textured 3D Mesh Generationcited by 69
  23. 2017Improved Adversarial Systems for 3D Object Generation and Reconstructioncited by 66
  24. 2017Learning to Segment Every Thingcited by 64
  25. 2018FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discoverycited by 58
  26. 2018Learning Descriptor Networks for 3D Shape Synthesis and Analysiscited by 56
  27. 2018Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Predictioncited by 52
  28. 2020Convolutional Generation of Textured 3D Meshescited by 52
  29. 2019Learning single-image 3D reconstruction by generative modelling of shape, pose and shadingcited by 43
  30. 2017LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generationcited by 36
  31. 20173D Object Reconstruction from a Single Depth View with Adversarial Learningcited by 26
  32. 2017SilNet : Single- and Multi-View Reconstruction by Learning from Silhouettescited by 24
  33. 2017Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Imagecited by 20
  34. 2017Weakly supervised 3D Reconstruction with Adversarial Constraintcited by 13
  35. 2019Controlling Style and Semantics in Weakly-Supervised Image Generationcited by 11
  36. 2018Structure-Aware Shape Synthesiscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 9; 2023: 18; 2024: 12; 2025: 5.

44 papers.

Sort:

By area:

  1. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  2. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  3. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  4. 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 324
  5. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
  6. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
  7. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  8. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  9. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  10. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  11. 2024Advances in 3D Generation: A Surveycited by 62
  12. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
32 more
  1. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  2. 2024TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusioncited by 51
  3. 2022Learning to Generate 3D Shapes from a Single Examplecited by 31
  4. 2023Mesh2Tex: Generating Mesh Textures from Image Queriescited by 31
  5. 2022Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversioncited by 30
  6. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  7. 2023Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shapecited by 29
  8. 2024Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splattingcited by 26
  9. 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wildcited by 25
  10. 2024ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplarscited by 14
  11. 2024PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformerscited by 14
  12. 2023ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Renderingcited by 13
  13. 2023Pushing the Limits of 3D Shape Generation at Scalecited by 11
  14. 2023Single Mesh Diffusion Models with Field Latents for Texture Generationcited by 11
  15. 2023TUVF: Learning Generalizable Texture UV Radiance Fieldscited by 9
  16. 2024SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Imagecited by 9
  17. 2023MVControl: Adding Conditional Control to Multi-view Diffusion for Controllable Text-to-3D Generationcited by 8
  18. 2023ET3D: Efficient Text-to-3D Generation via Multi-View Distillationcited by 6
  19. 2023TPA3D: Triplane Attention for Fast Text-to-3D Generationcited by 6
  20. 2023TEGLO: High Fidelity Canonical Texture Mapping from Single-View Imagescited by 4
  21. 2025PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priorscited by 4
  22. 2022Texture Generation Using A Graph Generative Adversarial Network And Differentiable Renderingcited by 3
  23. 2023Consistent Mesh Diffusioncited by 3
  24. 2023GET3D--: Learning GET3D from Unconstrained Image Collectionscited by 2
  25. 2025GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfercited by 2
  26. 2025Neural USD: An object-centric framework for iterative editing and controlcited by 2
  27. 2022Learning-based Monocular 3D Reconstruction of Birds: A Contemporary Surveycited by 1
  28. 2023Few-shot 3D Shape Generationcited by 1
  29. 2024Component Selection for Craft Assembly Taskscited by 1
  30. 2025GOATex: Geometry & Occlusion-Aware Texturingcited by 1
  31. 20243DTextureTransformer: Geometry Aware Texture Generation for Arbitrary Mesh Topologycited by 0
  32. 2024Texture- and Shape-based Adversarial Attacks for Overhead Image Vehicle Detectioncited by 0