Convolutional Generation of Textured 3D Meshes
Not on the map. Why?
Abstract (arXiv)
While recent generative models for 2D images achieve impressive visual results, they clearly lack the ability to perform 3D reasoning. This heavily restricts the degree of control over generated objects as well as the possible applications of such models. In this work, we bridge this gap by leveraging recent advances in differentiable rendering. We design a framework that can generate triangle meshes and associated high-resolution texture maps, using only 2D supervision from single-view natural images. A key contribution of our work is the encoding of the mesh and texture as 2D representations, which are semantically aligned and can be easily modeled by a 2D convolutional GAN. We demonstrate the efficacy of our method on Pascal3D+ Cars and CUB, both in an unconditional setting and in settings where the model is conditioned on class labels, attributes, and text. Finally, we propose an evaluation methodology that assesses the mesh and texture quality separately.
Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jun 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 5; 2022: 12; 2023: 18; 2024: 9; 2025: 6; 2026: 2.
52 papers.
Sort:
By area:
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
- 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
- 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
- 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
- 2022EpiGRAF: Rethinking training of 3D GANscited by 108
- 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
- 2023Texture Generation on 3D Meshes with Point-UV Diffusioncited by 63
- 2024Advances in 3D Generation: A Surveycited by 62
- 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
- 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
38 more
- 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priorscited by 43
- 2022AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesiscited by 39
- 2021Dual Contrastive Loss and Attention for GANscited by 30
- 2022Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversioncited by 30
- 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
- 2022Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistencycited by 26
- 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wildcited by 25
- 2024FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Headscited by 22
- 2021Representing 3D Shapes with Probabilistic Directed Distance Fieldscited by 21
- 2021View Generalization for Single Image Textured 3D Modelscited by 19
- 2022Hand-Object Interaction Image Generationcited by 19
- 2024AToM: Amortized Text-to-Mesh using 2D Diffusioncited by 18
- 20233DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Modelscited by 17
- 2022Fine Detailed Texture Learning for 3D Meshes with Generative Modelscited by 15
- 2024PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformerscited by 14
- 2023ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Renderingcited by 13
- 2023Progressive Learning of 3D Reconstruction Network from 2D GAN Datacited by 10
- 2023TUVF: Learning Generalizable Texture UV Radiance Fieldscited by 9
- 2024TexRO: Generating Delicate Textures of 3D Models by Recursive Optimizationcited by 9
- 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
- 2021Benchmarking the Robustness of Instance Segmentation Modelscited by 8
- 2022Monocular 3D Object Reconstruction with GAN Inversioncited by 8
- 2024Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Imagescited by 5
- 2023Differentiable Shadow Mapping for Efficient Inverse Graphicscited by 4
- 20223D-GIF: 3D-Controllable Object Generation via Implicit Factorized Representationscited by 3
- 2022Texture Generation Using A Graph Generative Adversarial Network And Differentiable Renderingcited by 3
- 2023A Review of Deep Learning-Powered Mesh Reconstruction Methodscited by 3
- 2023Mesh Watermark Removal Attack and Mitigation: A Novel Perspective of Function Spacecited by 3
- 2023GET3D--: Learning GET3D from Unconstrained Image Collectionscited by 2
- 2024Identity Preserving 3D Head Stylization with Multiview Score Distillationcited by 2
- 2025Exploring Kernel Transformations for Implicit Neural Representationscited by 2
- 2025Neural USD: An object-centric framework for iterative editing and controlcited by 2
- 2023Few-shot 3D Shape Generationcited by 1
- 2023Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Artcited by 1
- 2025GOATex: Geometry & Occlusion-Aware Texturingcited by 1
- 2026CustomTex: High-fidelity Indoor Scene Texturing via Multi-Reference Customizationcited by 1
- 2022Pruning-based Topology Refinement of 3D Mesh using a 2D Alpha Maskcited by 0
- 20233DMiner: Discovering Shapes from Large-Scale Unannotated Image Datasetscited by 0
50 most cited of 52 All 52 (JSON)
- 2026DynaTok: Token-Based 4D Reconstruction from Partial Point Clouds10 Jun 2026
- 2026CustomTex: High-fidelity Indoor Scene Texturing via Multi-Reference Customization19 Mar 2026
- 2025GOATex: Geometry & Occlusion-Aware Texturing28 Nov 2025
- 2025StyleFusion360: View-Consistent Head Stylization via Adaptive Style Modulation27 Nov 2025
- 2025Neural USD: An object-centric framework for iterative editing and control28 Oct 2025
- 2025Hadamard product in deep learning: Introduction, Advances and Challenges17 Apr 2025
- 2025Exploring Kernel Transformations for Implicit Neural Representations7 Apr 2025
- 2025MUSt3R: Multi-view Network for Stereo 3D Reconstruction3 Mar 2025
- 2024FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Heads23 Dec 2024
- 2024Identity Preserving 3D Head Stylization with Multiview Score Distillation20 Nov 2024
- 2024Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Images30 Sep 2024
- 2024PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformers18 Jul 2024
38 more
- 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models13 Jun 2024
- 2024TexRO: Generating Delicate Textures of 3D Models by Recursive Optimization22 Mar 2024
- 2024SPAD : Spatially Aware Multiview Diffusers7 Feb 2024
- 2024AToM: Amortized Text-to-Mesh using 2D Diffusion1 Feb 2024
- 2024Advances in 3D Generation: A Survey31 Jan 2024
- 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistency28 Dec 2023
- 2023DUSt3R: Geometric 3D Vision Made Easy21 Dec 2023
- 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors28 Nov 2023
- 2023Mesh Watermark Removal Attack and Mitigation: A Novel Perspective of Function Space18 Nov 2023
- 2023Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art16 Nov 2023
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov 2023
- 20233DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Models9 Nov 2023
- 20233DMiner: Discovering Shapes from Large-Scale Unannotated Image Datasets29 Oct 2023
- 2023Differentiable Shadow Mapping for Efficient Inverse Graphics21 Aug 2023
- 2023Texture Generation on 3D Meshes with Point-UV Diffusion21 Aug 2023
- 2023GET3D--: Learning GET3D from Unconstrained Image Collections27 Jul 2023
- 2023ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Rendering8 Jun 2023
- 2023ATT3D: Amortized Text-to-3D Object Synthesis6 Jun 2023
- 2023Few-shot 3D Shape Generation19 May 2023
- 2023Progressive Learning of 3D Reconstruction Network from 2D GAN Data18 May 2023
- 2023TUVF: Learning Generalizable Texture UV Radiance Fields4 May 2023
- 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wild4 Apr 2023
- 2023A Review of Deep Learning-Powered Mesh Reconstruction Methods6 Mar 2023
- 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec 2022
- 2022Hand-Object Interaction Image Generation28 Nov 2022
- 2022Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion21 Nov 2022
- 2022Pruning-based Topology Refinement of 3D Mesh using a 2D Alpha Mask17 Oct 2022
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images22 Sep 2022
- 2022Monocular 3D Object Reconstruction with GAN Inversion20 Jul 2022
- 2022EpiGRAF: Rethinking training of 3D GANs21 Jun 2022
- 2022Texture Generation Using A Graph Generative Adversarial Network And Differentiable Rendering17 Jun 2022
- 2022Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency21 Apr 2022
- 2022AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis6 Apr 2022
- 2022Fine Detailed Texture Learning for 3D Meshes with Generative Models17 Mar 2022
- 20223D-GIF: 3D-Controllable Object Generation via Implicit Factorized Representations12 Mar 2022
- 2021Representing 3D Shapes with Probabilistic Directed Distance Fields10 Dec 2021
- 2021Benchmarking the Robustness of Instance Segmentation Models2 Sep 2021
- 2021View Generalization for Single Image Textured 3D Models10 Jun 2021
50 newest of 52 All 52 (JSON)
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
- 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
- 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
- 2023Texture Generation on 3D Meshes with Point-UV Diffusioncited by 63
- 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priorscited by 43
- 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
- 20233DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Modelscited by 17
- 2025GOATex: Geometry & Occlusion-Aware Texturingcited by 1
- 2025GOATex: Geometry & Occlusion-Aware Texturing28 Nov 2025
- 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models13 Jun 2024
- 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistency28 Dec 2023
- 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors28 Nov 2023
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov 2023
- 20233DStyle-Diffusion: Pursuing Fine-grained Text-driven 3D Stylization with 2D Diffusion Models9 Nov 2023
- 2023Texture Generation on 3D Meshes with Point-UV Diffusion21 Aug 2023
- 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec 2022
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
- 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
- 2022EpiGRAF: Rethinking training of 3D GANscited by 108
- 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
- 2024Advances in 3D Generation: A Surveycited by 62
- 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
- 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
- 2022AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesiscited by 39
- 2021Dual Contrastive Loss and Attention for GANscited by 30
- 2022Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversioncited by 30
- 2022Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistencycited by 26
12 most cited of 44 All 44 (JSON)
- 2026DynaTok: Token-Based 4D Reconstruction from Partial Point Clouds10 Jun 2026
- 2026CustomTex: High-fidelity Indoor Scene Texturing via Multi-Reference Customization19 Mar 2026
- 2025StyleFusion360: View-Consistent Head Stylization via Adaptive Style Modulation27 Nov 2025
- 2025Neural USD: An object-centric framework for iterative editing and control28 Oct 2025
- 2025Hadamard product in deep learning: Introduction, Advances and Challenges17 Apr 2025
- 2025Exploring Kernel Transformations for Implicit Neural Representations7 Apr 2025
- 2025MUSt3R: Multi-view Network for Stereo 3D Reconstruction3 Mar 2025
- 2024FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Heads23 Dec 2024
- 2024Identity Preserving 3D Head Stylization with Multiview Score Distillation20 Nov 2024
- 2024Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Images30 Sep 2024
- 2024PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformers18 Jul 2024
- 2024TexRO: Generating Delicate Textures of 3D Models by Recursive Optimization22 Mar 2024
12 newest of 44 All 44 (JSON)