# Fine-Grained Image Style Transfer with Visual Transformers

arXiv: https://arxiv.org/abs/2210.05176 · HTML: https://mlsanity.com/paper/2210.05176/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 61 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [3D Object Detection with Pointformer](https://mlsanity.com/paper/2012.11409/) (2020)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer](https://mlsanity.com/paper/2108.03647/) (2021)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows](https://mlsanity.com/paper/2103.16877/) (2021)
- [Coherent Online Video Style Transfer](https://mlsanity.com/paper/1703.09211/) (2017)
- [Fast Patch-based Style Transfer of Arbitrary Style](https://mlsanity.com/paper/1612.04337/) (2016)
- [StyleBank: An Explicit Representation for Neural Image Style Transfer](https://mlsanity.com/paper/1703.09210/) (2017)
- [Arbitrary Video Style Transfer via Multi-Channel Correlation](https://mlsanity.com/paper/2009.08003/) (2020)

## Built on by (most cited first, 11 of 11)

- [Measuring Style Similarity in Diffusion Models](https://mlsanity.com/paper/2404.01292/) (2024)
- [Z^*: Zero-shot Style Transfer via Attention Rearrangement](https://mlsanity.com/paper/2311.16491/) (2023)
- [SaMam: Style-aware State Space Model for Arbitrary Image Style Transfer](https://mlsanity.com/paper/2503.15934/) (2025)
- [Learning Data-Driven Vector-Quantized Degradation Model for Animation Video Super-Resolution](https://mlsanity.com/paper/2303.09826/) (2023)
- [SPAST: Arbitrary Style Transfer with Style Priors via Pre-trained Large-scale Model](https://mlsanity.com/paper/2505.08695/) (2025)
- [Training-free Stylized Text-to-Image Generation with Fast Inference](https://mlsanity.com/paper/2505.19063/) (2025)
- [Semantix: An Energy Guided Sampler for Semantic Style Transfer](https://mlsanity.com/paper/2503.22344/) (2025)
- [EXTRACTER: Efficient Texture Matching with Attention and Gradient Enhancing for Large Scale Image Super Resolution](https://mlsanity.com/paper/2310.01379/) (2023)
- [Bridging Text and Image for Artist Style Transfer via Contrastive Learning](https://mlsanity.com/paper/2410.09566/) (2024)
- [Pluggable Style Representation Learning for Multi-Style Transfer](https://mlsanity.com/paper/2503.20368/) (2025)
- [MAST: Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfer](https://mlsanity.com/paper/2604.12281/) (2026)

## Abstract (arXiv)

With the development of the convolutional neural network, image style transfer has drawn increasing attention. However, most existing approaches adopt a global feature transformation to transfer style patterns into content images (e.g., AdaIN and WCT). Such a design usually destroys the spatial information of the input images and fails to transfer fine-grained style patterns into style transfer results. To solve this problem, we propose a novel STyle TRansformer (STTR) network which breaks both content and style images into visual tokens to achieve a fine-grained style transformation. Specifically, two attention mechanisms are adopted in our STTR. We first propose to use self-attention to encode content and style tokens such that similar tokens can be grouped and learned together. We then adopt cross-attention between content and style tokens that encourages fine-grained style transformations. To compare STTR with existing approaches, we conduct user studies on Amazon Mechanical Turk (AMT), which are carried out with 50 human subjects with 1,000 votes in total. Extensive evaluations demonstrate the effectiveness and efficiency of the proposed STTR in generating visually pleasing style transfer results.