# TextureGAN: Controlling Deep Image Synthesis with Texture Patches

arXiv: https://arxiv.org/abs/1706.02823 · HTML: https://mlsanity.com/paper/1706.02823/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2017.

## Built on by (most cited first, 30 of 47)

- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [Text-Guided Texturing by Synchronized Multi-View Diffusion](https://mlsanity.com/paper/2311.12891/) (2023)
- [InfinityGAN: Towards Infinite-Pixel Image Synthesis](https://mlsanity.com/paper/2104.03963/) (2021)
- [Language-Driven Image Style Transfer](https://mlsanity.com/paper/2106.00178/) (2021)
- [VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers](https://mlsanity.com/paper/2104.06757/) (2021)
- [Semi-supervised Synthesis of High-Resolution Editable Textures for 3D Humans](https://mlsanity.com/paper/2103.17266/) (2021)
- [Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer](https://mlsanity.com/paper/2108.07520/) (2021)
- [CoGS: Controllable Generation and Search from Sketch and Style](https://mlsanity.com/paper/2203.09554/) (2022)
- [Facial-Sketch Synthesis: A New Challenge](https://mlsanity.com/paper/2112.15439/) (2021)
- [FashionTex: Controllable Virtual Try-on with Text and Texture](https://mlsanity.com/paper/2305.04451/) (2023)
- [Model Extraction and Defenses on Generative Adversarial Networks](https://mlsanity.com/paper/2101.02069/) (2021)
- [SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution](https://mlsanity.com/paper/2201.04364/) (2022)
- [An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering](https://mlsanity.com/paper/2103.03390/) (2021)
- [SemIE: Semantically-aware Image Extrapolation](https://mlsanity.com/paper/2108.13702/) (2021)
- [Focusing on Persons: Colorizing Old Images Learning from Modern Historical Movies](https://mlsanity.com/paper/2108.06515/) (2021)
- [ColorFlow: Retrieval-Augmented Image Sequence Colorization](https://mlsanity.com/paper/2412.11815/) (2024)
- [Cobra: Efficient Line Art COlorization with BRoAder References](https://mlsanity.com/paper/2504.12240/) (2025)
- [Patch of Invisibility: Naturalistic Physical Black-Box Adversarial Attacks on Object Detectors](https://mlsanity.com/paper/2303.04238/) (2023)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)
- [Exemplar-based Pattern Synthesis with Implicit Periodic Field Network](https://mlsanity.com/paper/2204.01671/) (2022)
- [Texture Generation Using A Graph Generative Adversarial Network And Differentiable Rendering](https://mlsanity.com/paper/2206.08547/) (2022)
- [Improving Sketch Colorization using Adversarial Segmentation Consistency](https://mlsanity.com/paper/2301.08590/) (2023)
- [Directional Texture Editing for 3D Models](https://mlsanity.com/paper/2309.14872/) (2023)
- [Retargeting Visual Data with Deformation Fields](https://mlsanity.com/paper/2311.13297/) (2023)
- [Enhanced Multi-Scale Cross-Attention for Person Image Generation](https://mlsanity.com/paper/2501.08900/) (2025)
- [Adversarial Segmentation Loss for Sketch Colorization](https://mlsanity.com/paper/2102.06192/) (2021)
- [FBC-GAN: Diverse and Flexible Image Synthesis via Foreground-Background Composition](https://mlsanity.com/paper/2107.03166/) (2021)
- [TriTex: Learning Texture from a Single Mesh via Triplane Semantic Features](https://mlsanity.com/paper/2503.16630/) (2025)
- [UltraZoom: Generating Gigapixel Images from Regular Photos](https://mlsanity.com/paper/2506.13756/) (2025)

## Abstract (arXiv)

In this paper, we investigate deep image synthesis guided by sketch, color, and texture. Previous image synthesis methods can be controlled by sketch and color strokes but we are the first to examine texture control. We allow a user to place a texture patch on a sketch at arbitrary locations and scales to control the desired output texture. Our generative network learns to synthesize objects consistent with these texture suggestions. To achieve this, we develop a local texture loss in addition to adversarial and content loss to train the generative network. We conduct experiments using sketches generated from real images and textures sampled from a separate texture database and results show that our proposed algorithm is able to generate plausible images that are faithful to user controls. Ablation studies show that our proposed pipeline can generate more realistic images than adapting existing methods directly.