# DivSwapper: Towards Diversified Patch-based Arbitrary Style Transfer

arXiv: https://arxiv.org/abs/2101.06381 · HTML: https://mlsanity.com/paper/2101.06381/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 16 Jan 2021.

## Builds on (most cited first, 26 of 26)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Neural Style Transfer: A Review](https://mlsanity.com/paper/1705.04058/) (2017)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [Style Transfer by Relaxed Optimal Transport and Self-Similarity](https://mlsanity.com/paper/1904.12785/) (2019)
- [Demystifying Neural Style Transfer](https://mlsanity.com/paper/1701.01036/) (2017)
- [Visual Attribute Transfer through Deep Image Analogy](https://mlsanity.com/paper/1705.01088/) (2017)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis](https://mlsanity.com/paper/1601.04589/) (2016)
- [Fast Patch-based Style Transfer of Arbitrary Style](https://mlsanity.com/paper/1612.04337/) (2016)
- [StyleBank: An Explicit Representation for Neural Image Style Transfer](https://mlsanity.com/paper/1703.09210/) (2017)
- [Dynamic Instance Normalization for Arbitrary Style Transfer](https://mlsanity.com/paper/1911.06953/) (2019)
- [Arbitrary Style Transfer with Deep Feature Reshuffle](https://mlsanity.com/paper/1805.04103/) (2018)
- [Arbitrary Style Transfer via Multi-Adaptation Network](https://mlsanity.com/paper/2005.13219/) (2020)
- [Diversified Texture Synthesis with Feed-forward Networks](https://mlsanity.com/paper/1703.01664/) (2017)
- [Attention-aware Multi-stroke Style Transfer](https://mlsanity.com/paper/1901.05127/) (2019)
- [A Closed-form Solution to Universal Style Transfer](https://mlsanity.com/paper/1906.00668/) (2019)
- [Diversified Arbitrary Style Transfer via Deep Feature Perturbation](https://mlsanity.com/paper/1909.08223/) (2019)
- [Multimodal Style Transfer via Graph Cuts](https://mlsanity.com/paper/1904.04443/) (2019)
- [Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks](https://mlsanity.com/paper/1603.01768/) (2016)

## Built on by (most cited first, 11 of 11)

- [StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models](https://mlsanity.com/paper/2308.07863/) (2023)
- [AesUST: Towards Aesthetic-Enhanced Universal Style Transfer](https://mlsanity.com/paper/2208.13016/) (2022)
- [MicroAST: Towards Super-Fast Ultra-Resolution Arbitrary Style Transfer](https://mlsanity.com/paper/2211.15313/) (2022)
- [QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity](https://mlsanity.com/paper/2212.10431/) (2022)
- [TSSAT: Two-Stage Statistics-Aware Transformation for Artistic Style Transfer](https://mlsanity.com/paper/2309.06004/) (2023)
- [Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation](https://mlsanity.com/paper/2403.08294/) (2024)
- [Lightweight texture transfer based on texture feature preset](https://mlsanity.com/paper/2306.16846/) (2023)
- [Color and Texture Dual Pipeline Lightweight Style Transfer](https://mlsanity.com/paper/2310.01321/) (2023)
- [CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer](https://mlsanity.com/paper/2602.14464/) (2026)
- [Retinex-guided Channel-grouping based Patch Swap for Arbitrary Style Transfer](https://mlsanity.com/paper/2309.10528/) (2023)
- [Dual Pipeline Style Transfer with Input Distribution Differentiation](https://mlsanity.com/paper/2311.05432/) (2023)

## Abstract (arXiv)

Gram-based and patch-based approaches are two important research lines of style transfer. Recent diversified Gram-based methods have been able to produce multiple and diverse stylized outputs for the same content and style images. However, as another widespread research interest, the diversity of patch-based methods remains challenging due to the stereotyped style swapping process based on nearest patch matching. To resolve this dilemma, in this paper, we dive into the crux of existing patch-based methods and propose a universal and efficient module, termed DivSwapper, for diversified patch-based arbitrary style transfer. The key insight is to use an essential intuition that neural patches with higher activation values could contribute more to diversity. Our DivSwapper is plug-and-play and can be easily integrated into existing patch-based and Gram-based methods to generate diverse results for arbitrary styles. We conduct theoretical analyses and extensive experiments to demonstrate the effectiveness of our method, and compared with state-of-the-art algorithms, it shows superiority in diversity, quality, and efficiency.