mlsanityData to 7 Oct 2026

Neural Photometry-guided Visual Attribute Transfer

arXiv 2112.02520 ↗First posted 5 Dec 2021cs.CV cs.AI cs.GRcited by 6

Not on the map. Why?

Abstract (arXiv)

We present a deep learning-based method for propagating spatially-varying visual material attributes (e.g. texture maps or image stylizations) to larger samples of the same or similar materials. For training, we leverage images of the material taken under multiple illuminations and a dedicated data augmentation policy, making the transfer robust to novel illumination conditions and affine deformations. Our model relies on a supervised image-to-image translation framework and is agnostic to the transferred domain; we showcase a semantic segmentation, a normal map, and a stylization. Following an image analogies approach, the method only requires the training data to contain the same visual structures as the input guidance. Our approach works at interactive rates, making it suitable for material edit applications. We thoroughly evaluate our learning methodology in a controlled setup providing quantitative measures of performance. Last, we demonstrate that training the model on a single material is enough to generalize to materials of the same type without the need for massive datasets.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 48 of its 106 references that are in the map. First posted on arXiv 5 Dec 2021.

Builds on

48 of 106 references.

References · 48

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  8. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  9. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  10. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  11. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  12. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
36 more
  1. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  2. 2017Deep Image Priorcited by 1,097
  3. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  4. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  5. 2015A Neural Algorithm of Artistic Stylecited by 480
  6. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  7. 2017Universal Style Transfer via Feature Transformscited by 352
  8. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  9. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  10. 2017Toward Multimodal Image-to-Image Translationcited by 295
  11. 2015A note on the evaluation of generative modelscited by 291
  12. 2017"Zero-Shot" Super-Resolution using Deep Internal Learningcited by 285
  13. 2017Data Augmentation Generative Adversarial Networkscited by 278
  14. 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
  15. 2017Neural Style Transfer: A Reviewcited by 199
  16. 2019Few-shot Video-to-Video Synthesiscited by 196
  17. 2015Weakly Supervised Deep Detection Networkscited by 183
  18. 2017Visual Attribute Transfer through Deep Image Analogycited by 123
  19. 2018Single-Image SVBRDF Capture with a Rendering-Aware Deep Networkcited by 116
  20. 2016Controlling Perceptual Factors in Neural Style Transfercited by 99
  21. 2017Coherent Online Video Style Transfercited by 96
  22. 2017StyleBank: An Explicit Representation for Neural Image Style Transfercited by 89
  23. 2018Deep Exemplar-based Colorizationcited by 82
  24. 2018Arbitrary Style Transfer with Deep Feature Reshufflecited by 75
  25. 2019Deep Exemplar-based Video Colorizationcited by 75
  26. 2018Non-Stationary Texture Synthesis by Adversarial Expansioncited by 66
  27. 2020Learning Invariances in Neural Networkscited by 52
  28. 2018Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networkscited by 51
  29. 2019Flexible SVBRDF Capture with a Multi-Image Deep Networkcited by 50
  30. 2020Interactive Video Stylization Using Few-Shot Patch-Based Trainingcited by 43
  31. 2020MaterialGAN: Reflectance Capture using a Generative SVBRDF Modelcited by 33
  32. 2017Quantifying Translation-Invariance in Convolutional Neural Networkscited by 30
  33. 2020Guided Fine-Tuning for Large-Scale Material Transfercited by 27
  34. 2016Style-Transfer via Texture-Synthesiscited by 17
  35. 2019TileGAN: Synthesis of Large-Scale Non-Homogeneous Texturescited by 15
  36. 2020Structural-analogy from a Single Image Paircited by 14

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 1.

6 papers.

  1. 2024A survey of synthetic data augmentation methods in computer visioncited by 34
  2. 2022TileGen: Tileable, Controllable Material Generation and Capturecited by 27
  3. 2023UMat: Uncertainty-Aware Single Image High Resolution Material Capturecited by 11
  4. 2023NeuBTF: Neural fields for BTF encoding and transfercited by 5
  5. 2024PoseBench: Benchmarking the Robustness of Pose Estimation Models under Corruptionscited by 4
  6. 2025Single-image Reflectance and Transmittance Estimation from Any Flatbed Scannercited by 1