# AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis

arXiv: https://arxiv.org/abs/2204.03105 · HTML: https://mlsanity.com/paper/2204.03105/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 48 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Texture Fields: Learning Texture Representations in Function Space](https://mlsanity.com/paper/1905.07259/) (2019)
- [BSP-Net: Generating Compact Meshes via Binary Space Partitioning](https://mlsanity.com/paper/1911.06971/) (2019)
- [Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild](https://mlsanity.com/paper/1911.11130/) (2019)
- [Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings](https://mlsanity.com/paper/1803.08495/) (2018)
- [Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondence](https://mlsanity.com/paper/2011.13650/) (2020)
- [SDM-NET: Deep Generative Network for Structured Deformable Mesh](https://mlsanity.com/paper/1908.04520/) (2019)
- [i3DMM: Deep Implicit 3D Morphable Model of Human Heads](https://mlsanity.com/paper/2011.14143/) (2020)
- [Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering](https://mlsanity.com/paper/2010.09125/) (2020)
- [Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research](https://mlsanity.com/paper/1911.05063/) (2019)
- [NeuTex: Neural Texture Mapping for Volumetric Neural Rendering](https://mlsanity.com/paper/2103.00762/) (2021)
- [BAE-NET: Branched Autoencoder for Shape Co-Segmentation](https://mlsanity.com/paper/1903.11228/) (2019)
- [Learning Deformable Tetrahedral Meshes for 3D Reconstruction](https://mlsanity.com/paper/2011.01437/) (2020)
- [Leveraging 2D Data to Learn Textured 3D Mesh Generation](https://mlsanity.com/paper/2004.04180/) (2020)
- [Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs](https://mlsanity.com/paper/2011.00844/) (2020)
- [3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations](https://mlsanity.com/paper/2108.12958/) (2021)

## Built on by (most cited first, 30 of 39)

- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models](https://mlsanity.com/paper/2312.13913/) (2023)
- [TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models](https://mlsanity.com/paper/2310.13772/) (2023)
- [TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion](https://mlsanity.com/paper/2401.09416/) (2024)
- [Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing](https://mlsanity.com/paper/2403.10050/) (2024)
- [OR-NeRF: Object Removing from 3D Scenes Guided by Multiview Segmentation with Neural Radiance Fields](https://mlsanity.com/paper/2305.10503/) (2023)
- [Towards Garment Sewing Pattern Reconstruction from a Single Image](https://mlsanity.com/paper/2311.04218/) (2023)
- [Nuvo: Neural UV Mapping for Unruly 3D Representations](https://mlsanity.com/paper/2312.05283/) (2023)
- [TexPainter: Generative Mesh Texturing with Multi-view Consistency](https://mlsanity.com/paper/2406.18539/) (2024)
- [CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulation](https://mlsanity.com/paper/2311.01447/) (2023)
- [EASI-Tex: Edge-Aware Mesh Texturing from Single Image](https://mlsanity.com/paper/2405.17393/) (2024)
- [GenesisTex: Adapting Image Denoising Diffusion to Texture Space](https://mlsanity.com/paper/2403.17782/) (2024)
- [RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting](https://mlsanity.com/paper/2406.02461/) (2024)
- [StyleTex: Style Image-Guided Texture Generation for 3D Models](https://mlsanity.com/paper/2411.00399/) (2024)
- [ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Rendering](https://mlsanity.com/paper/2306.04889/) (2023)
- [Single Mesh Diffusion Models with Field Latents for Texture Generation](https://mlsanity.com/paper/2312.09250/) (2023)
- [TUVF: Learning Generalizable Texture UV Radiance Fields](https://mlsanity.com/paper/2305.03040/) (2023)
- [TexRO: Generating Delicate Textures of 3D Models by Recursive Optimization](https://mlsanity.com/paper/2403.15009/) (2024)
- [PartUV: Part-Based UV Unwrapping of 3D Meshes](https://mlsanity.com/paper/2511.16659/) (2025)
- [VCD-Texture: Variance Alignment based 3D-2D Co-Denoising for Text-Guided Texturing](https://mlsanity.com/paper/2407.04461/) (2024)
- [FlexiTex: Enhancing Texture Generation via Visual Guidance](https://mlsanity.com/paper/2409.12431/) (2024)
- [GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation](https://mlsanity.com/paper/2409.18401/) (2024)
- [Compositional Neural Textures](https://mlsanity.com/paper/2404.12509/) (2024)
- [TEGLO: High Fidelity Canonical Texture Mapping from Single-View Images](https://mlsanity.com/paper/2303.13743/) (2023)
- [A Generative Multi-Resolution Pyramid and Normal-Conditioning 3D Cloth Draping](https://mlsanity.com/paper/2311.02700/) (2023)
- [PointDreamer: Zero-shot 3D Textured Mesh Reconstruction from Colored Point Cloud](https://mlsanity.com/paper/2406.15811/) (2024)
- [UV-free Texture Generation with Denoising and Geodesic Heat Diffusions](https://mlsanity.com/paper/2408.16762/) (2024)
- [GT2-GS: Geometry-aware Texture Transfer for Gaussian Splatting](https://mlsanity.com/paper/2505.15208/) (2025)
- [Learning Neural Implicit Representations with Surface Signal Parameterizations](https://mlsanity.com/paper/2211.00519/) (2022)
- [Zippo: Zipping Color and Transparency Distributions into a Single Diffusion Model](https://mlsanity.com/paper/2403.11077/) (2024)

## Abstract (arXiv)

In this paper, we address the problem of texture representation for 3D shapes for the challenging and underexplored tasks of texture transfer and synthesis. Previous works either apply spherical texture maps which may lead to large distortions, or use continuous texture fields that yield smooth outputs lacking details. We argue that the traditional way of representing textures with images and linking them to a 3D mesh via UV mapping is more desirable, since synthesizing 2D images is a well-studied problem. We propose AUV-Net which learns to embed 3D surfaces into a 2D aligned UV space, by mapping the corresponding semantic parts of different 3D shapes to the same location in the UV space. As a result, textures are aligned across objects, and can thus be easily synthesized by generative models of images. Texture alignment is learned in an unsupervised manner by a simple yet effective texture alignment module, taking inspiration from traditional works on linear subspace learning. The learned UV mapping and aligned texture representations enable a variety of applications including texture transfer, texture synthesis, and textured single view 3D reconstruction. We conduct experiments on multiple datasets to demonstrate the effectiveness of our method. Project page: https://nv-tlabs.github.io/AUV-NET.