# Non-Parametric Neural Style Transfer

arXiv: https://arxiv.org/abs/2108.12847 · HTML: https://mlsanity.com/paper/2108.12847/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 49 of its 128 references that are in the map.
- First posted on arXiv 29 Aug 2021.

## Builds on (most cited first, 30 of 49)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature](https://mlsanity.com/paper/1505.00855/) (2015)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [The Contextual Loss for Image Transformation with Non-Aligned Data](https://mlsanity.com/paper/1803.02077/) (2018)
- [Hypercolumns for Object Segmentation and Fine-grained Localization](https://mlsanity.com/paper/1411.5752/) (2014)
- [Visual Attribute Transfer through Deep Image Analogy](https://mlsanity.com/paper/1705.01088/) (2017)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows](https://mlsanity.com/paper/2103.16877/) (2021)
- [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis](https://mlsanity.com/paper/1601.04589/) (2016)
- [Controlling Perceptual Factors in Neural Style Transfer](https://mlsanity.com/paper/1611.07865/) (2016)
- [StyleBank: An Explicit Representation for Neural Image Style Transfer](https://mlsanity.com/paper/1703.09210/) (2017)
- [Arbitrary Style Transfer with Deep Feature Reshuffle](https://mlsanity.com/paper/1805.04103/) (2018)

## Abstract (arXiv)

It seems easy to imagine a photograph of the Eiffel Tower painted in the style of Vincent van Gogh's 'The Starry Night', but upon introspection it is difficult to precisely define what this would entail. What visual elements must an image contain to represent the 'content' of the Eiffel Tower? What visual elements of 'The Starry Night' are caused by van Gogh's 'style' rather than his decision to depict a village under the night sky? Precisely defining 'content' and 'style' is a central challenge of designing algorithms for artistic style transfer, algorithms which can recreate photographs using an artwork's style. My efforts defining these terms, and designing style transfer algorithms themselves, are the focus of this thesis. I will begin by proposing novel definitions of style and content based on optimal transport and self-similarity, and demonstrating how a style transfer algorithm based on these definitions generates outputs with improved visual quality. Then I will describe how the traditional texture-based definition of style can be expanded to include elements of geometry and proportion by jointly optimizing a keypoint-guided deformation field alongside the stylized output's pixels. Finally I will describe a framework inspired by both modern neural style transfer algorithms and traditional patch-based synthesis approaches which is fast, general, and offers state-of-the-art visual quality.