# GramGAN: Deep 3D Texture Synthesis From 2D Exemplars

arXiv: https://arxiv.org/abs/2006.16112 · HTML: https://mlsanity.com/paper/2006.16112/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2020.

## Built on by (most cited first, 18 of 18)

- [Aligning Latent and Image Spaces to Connect the Unconnectable](https://mlsanity.com/paper/2104.06954/) (2021)
- [Texture Generation on 3D Meshes with Point-UV Diffusion](https://mlsanity.com/paper/2308.10490/) (2023)
- [SinGRAF: Learning a 3D Generative Radiance Field for a Single Scene](https://mlsanity.com/paper/2211.17260/) (2022)
- [Mesh Neural Cellular Automata](https://mlsanity.com/paper/2311.02820/) (2023)
- [Texture Generation with Neural Cellular Automata](https://mlsanity.com/paper/2105.07299/) (2021)
- [TUVF: Learning Generalizable Texture UV Radiance Fields](https://mlsanity.com/paper/2305.03040/) (2023)
- [TexRO: Generating Delicate Textures of 3D Models by Recursive Optimization](https://mlsanity.com/paper/2403.15009/) (2024)
- [VCD-Texture: Variance Alignment based 3D-2D Co-Denoising for Text-Guided Texturing](https://mlsanity.com/paper/2407.04461/) (2024)
- [GANmouflage: 3D Object Nondetection with Texture Fields](https://mlsanity.com/paper/2201.07202/) (2022)
- [Towards Universal Texture Synthesis by Combining Texton Broadcasting with Noise Injection in StyleGAN-2](https://mlsanity.com/paper/2203.04221/) (2022)
- [Exemplar-based Pattern Synthesis with Implicit Periodic Field Network](https://mlsanity.com/paper/2204.01671/) (2022)
- [Texture Representation via Analysis and Synthesis with Generative Adversarial Networks](https://mlsanity.com/paper/2212.09983/) (2022)
- [STS-GAN: Can We Synthesize Solid Texture with High Fidelity from Arbitrary 2D Exemplar?](https://mlsanity.com/paper/2102.03973/) (2021)
- [Content-aware Tile Generation using Exterior Boundary Inpainting](https://mlsanity.com/paper/2409.14184/) (2024)
- [NeRF-Texture: Synthesizing Neural Radiance Field Textures](https://mlsanity.com/paper/2412.10004/) (2024)
- [3D microstructural generation from 2D images of cement paste using generative adversarial networks](https://mlsanity.com/paper/2204.01645/) (2022)
- [Automated segmentation and morphological characterization of placental histology images based on a single labeled image](https://mlsanity.com/paper/2210.03566/) (2022)
- [PolyCrysDiff: Controllable Generation of Three-Dimensional Computable Polycrystalline Material Structures](https://mlsanity.com/paper/2603.11695/) (2026)

## Abstract (arXiv)

We present a novel texture synthesis framework, enabling the generation of infinite, high-quality 3D textures given a 2D exemplar image. Inspired by recent advances in natural texture synthesis, we train deep neural models to generate textures by non-linearly combining learned noise frequencies. To achieve a highly realistic output conditioned on an exemplar patch, we propose a novel loss function that combines ideas from both style transfer and generative adversarial networks. In particular, we train the synthesis network to match the Gram matrices of deep features from a discriminator network. In addition, we propose two architectural concepts and an extrapolation strategy that significantly improve generalization performance. In particular, we inject both model input and condition into hidden network layers by learning to scale and bias hidden activations. Quantitative and qualitative evaluations on a diverse set of exemplars motivate our design decisions and show that our system performs superior to previous state of the art. Finally, we conduct a user study that confirms the benefits of our framework.