# Multispectral Texture Synthesis using RGB Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2410.16019 · HTML: https://mlsanity.com/paper/2410.16019/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 21 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [DiffusionSat: A Generative Foundation Model for Satellite Imagery](https://mlsanity.com/paper/2312.03606/) (2023)
- [Rethinking and Improving the Robustness of Image Style Transfer](https://mlsanity.com/paper/2104.05623/) (2021)
- [Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing](https://mlsanity.com/paper/2310.18653/) (2023)
- [Texture Synthesis with Spatial Generative Adversarial Networks](https://mlsanity.com/paper/1611.08207/) (2016)
- [HSIGene: A Foundation Model For Hyperspectral Image Generation](https://mlsanity.com/paper/2409.12470/) (2024)
- [Deep Learning Techniques for Hyperspectral Image Analysis in Agriculture: A Review](https://mlsanity.com/paper/2304.13880/) (2023)
- [Generating Synthetic Multispectral Satellite Imagery from Sentinel-2](https://mlsanity.com/paper/2012.03108/) (2020)
- [Texture Synthesis Using Shallow Convolutional Networks with Random Filters](https://mlsanity.com/paper/1606.00021/) (2016)
- [Towards Universal Texture Synthesis by Combining Texton Broadcasting with Noise Injection in StyleGAN-2](https://mlsanity.com/paper/2203.04221/) (2022)
- [A geometrically aware auto-encoder for multi-texture synthesis](https://mlsanity.com/paper/2302.01616/) (2023)

## Abstract (arXiv)

State-of-the-art RGB texture synthesis algorithms rely on style distances that are computed through statistics of deep features. These deep features are extracted by classification neural networks that have been trained on large datasets of RGB images. Extending such synthesis methods to multispectral images is not straightforward, since the pre-trained networks are designed for and have been trained on RGB images. In this work, we propose two solutions to extend these methods to multispectral imaging. Neither of them require additional training of the neural network from which the second order neural statistics are extracted. The first one consists in optimizing over batches of random triplets of spectral bands throughout training. The second one projects multispectral pixels onto a 3 dimensional space. We further explore the benefit of a color transfer operation upstream of the projection to avoid the potentially abnormal color distributions induced by the projection. Our experiments compare the performances of the various methods through different metrics. We demonstrate that they can be used to perform exemplar-based texture synthesis, achieve good visual quality and comes close to state-of-the art methods on RGB bands.