# Compositional Neural Textures

arXiv: https://arxiv.org/abs/2404.12509 · HTML: https://mlsanity.com/paper/2404.12509/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 64 references that are in the map.
- First posted on arXiv 18 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Neural Scene Graphs for Dynamic Scenes](https://mlsanity.com/paper/2011.10379/) (2020)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022)
- [Style Transfer by Relaxed Optimal Transport and Self-Similarity](https://mlsanity.com/paper/1904.12785/) (2019)
- [SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos](https://mlsanity.com/paper/2206.07764/) (2022)
- [Superpixel Segmentation with Fully Convolutional Networks](https://mlsanity.com/paper/2003.12929/) (2020)
- [Non-Stationary Texture Synthesis by Adversarial Expansion](https://mlsanity.com/paper/1805.04487/) (2018)
- [Learning Texture Manifolds with the Periodic Spatial GAN](https://mlsanity.com/paper/1705.06566/) (2017)
- [Learning a Neural 3D Texture Space from 2D Exemplars](https://mlsanity.com/paper/1912.04158/) (2019)
- [Deep TEN: Texture Encoding Network](https://mlsanity.com/paper/1612.02844/) (2016)
- [AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis](https://mlsanity.com/paper/2204.03105/) (2022)
- [Deep Texture Manifold for Ground Terrain Recognition](https://mlsanity.com/paper/1803.10896/) (2018)
- [General Image-to-Image Translation with One-Shot Image Guidance](https://mlsanity.com/paper/2307.14352/) (2023)
- [SPAGHETTI: Editing Implicit Shapes Through Part Aware Generation](https://mlsanity.com/paper/2201.13168/) (2022)
- [Generating Procedural Materials from Text or Image Prompts](https://mlsanity.com/paper/2304.13172/) (2023)
- [GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation](https://mlsanity.com/paper/2112.01036/) (2021)
- [Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture](https://mlsanity.com/paper/1901.03447/) (2019)
- [TileGAN: Synthesis of Large-Scale Non-Homogeneous Textures](https://mlsanity.com/paper/1904.12795/) (2019)
- [Diffusion-based Holistic Texture Rectification and Synthesis](https://mlsanity.com/paper/2309.14759/) (2023)

## Built on by (most cited first, 5 of 5)

- [Image-GS: Content-Adaptive Image Representation via 2D Gaussians](https://mlsanity.com/paper/2407.01866/) (2024)
- [TextureSAM: Towards a Texture Aware Foundation Model for Segmentation](https://mlsanity.com/paper/2505.16540/) (2025)
- [T-MLP: Tailed Multi-Layer Perceptron for Level-of-Detail Signal Representation](https://mlsanity.com/paper/2509.00066/) (2025)
- [Sub-Semantic Image Segmentation](https://mlsanity.com/paper/2606.14754/) (2026)
- [Texture Interpolation as Texton Matching](https://mlsanity.com/paper/2610.18655/) (2026)

## Abstract (arXiv)

Texture plays a vital role in enhancing visual richness in both real photographs and computer-generated imagery. However, the process of editing textures often involves laborious and repetitive manual adjustments of textons, which are the recurring local patterns that characterize textures. This work introduces a fully unsupervised approach for representing textures using a compositional neural model that captures individual textons. We represent each texton as a 2D Gaussian function whose spatial support approximates its shape, and an associated feature that encodes its detailed appearance. By modeling a texture as a discrete composition of Gaussian textons, the representation offers both expressiveness and ease of editing. Textures can be edited by modifying the compositional Gaussians within the latent space, and new textures can be efficiently synthesized by feeding the modified Gaussians through a generator network in a feed-forward manner. This approach enables a wide range of applications, including transferring appearance from an image texture to another image, diversifying textures,texture interpolation, revealing/modifying texture variations, edit propagation, texture animation, and direct texton manipulation. The proposed approach contributes to advancing texture analysis, modeling, and editing techniques, and opens up new possibilities for creating visually appealing images with controllable textures.