# StyTr2: Image Style Transfer with Transformers

arXiv: https://arxiv.org/abs/2105.14576 · HTML: https://mlsanity.com/paper/2105.14576/

## Key facts

- Cited by 187 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 66 references that are in the map.
- First posted on arXiv 30 May 2021.

## Builds on (most cited first, 30 of 51)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020)
- [Vision Transformers are Robust Learners](https://mlsanity.com/paper/2105.07581/) (2021)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [How Much Position Information Do Convolutional Neural Networks Encode?](https://mlsanity.com/paper/2001.08248/) (2020)

## Built on by (most cited first, 30 of 187)

- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [StyleDrop: Text-to-Image Generation in Any Style](https://mlsanity.com/paper/2306.00983/) (2023)
- [Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer](https://mlsanity.com/paper/2312.09008/) (2023)
- [StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models](https://mlsanity.com/paper/2308.07863/) (2023)
- [Implicit Style-Content Separation using B-LoRA](https://mlsanity.com/paper/2403.14572/) (2024)
- [Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning](https://mlsanity.com/paper/2205.09542/) (2022)
- [CSGO: Content-Style Composition in Text-to-Image Generation](https://mlsanity.com/paper/2408.16766/) (2024)
- [DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations](https://mlsanity.com/paper/2403.06951/) (2024)
- [StyleShot: A Snapshot on Any Style](https://mlsanity.com/paper/2407.01414/) (2024)
- [UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Models](https://mlsanity.com/paper/2402.11846/) (2024)
- [ArtFID: Quantitative Evaluation of Neural Style Transfer](https://mlsanity.com/paper/2207.12280/) (2022)
- [Photoswap: Personalized Subject Swapping in Images](https://mlsanity.com/paper/2305.18286/) (2023)
- [DiffStyler: Controllable Dual Diffusion for Text-Driven Image Stylization](https://mlsanity.com/paper/2211.10682/) (2022)
- [ControlStyle: Text-Driven Stylized Image Generation Using Diffusion Priors](https://mlsanity.com/paper/2311.05463/) (2023)
- [ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptation](https://mlsanity.com/paper/2312.02109/) (2023)
- [StyleAdapter: A Unified Stylized Image Generation Model](https://mlsanity.com/paper/2309.01770/) (2023)
- [Neural Preset for Color Style Transfer](https://mlsanity.com/paper/2303.13511/) (2023)
- [AesPA-Net: Aesthetic Pattern-Aware Style Transfer Networks](https://mlsanity.com/paper/2307.09724/) (2023)
- [Attention Distillation: A Unified Approach to Visual Characteristics Transfer](https://mlsanity.com/paper/2502.20235/) (2025)
- [StyleTokenizer: Defining Image Style by a Single Instance for Controlling Diffusion Models](https://mlsanity.com/paper/2409.02543/) (2024)
- [General Image-to-Image Translation with One-Shot Image Guidance](https://mlsanity.com/paper/2307.14352/) (2023)
- [Draw Your Art Dream: Diverse Digital Art Synthesis with Multimodal Guided Diffusion](https://mlsanity.com/paper/2209.13360/) (2022)
- [DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2309.06933/) (2023)
- [FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models](https://mlsanity.com/paper/2401.15636/) (2024)
- [MicroAST: Towards Super-Fast Ultra-Resolution Arbitrary Style Transfer](https://mlsanity.com/paper/2211.15313/) (2022)
- [QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity](https://mlsanity.com/paper/2212.10431/) (2022)
- [SRFormerV2: Taking a Closer Look at Permuted Self-Attention for Image Super-Resolution](https://mlsanity.com/paper/2303.09735/) (2023)
- [MoST: Motion Style Transformer between Diverse Action Contents](https://mlsanity.com/paper/2403.06225/) (2024)
- [Any-to-Any Style Transfer: Making Picasso and Da Vinci Collaborate](https://mlsanity.com/paper/2304.09728/) (2023)
- [InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser](https://mlsanity.com/paper/2311.15040/) (2023)

## Abstract (arXiv)

The goal of image style transfer is to render an image with artistic features guided by a style reference while maintaining the original content. Owing to the locality in convolutional neural networks (CNNs), extracting and maintaining the global information of input images is difficult. Therefore, traditional neural style transfer methods face biased content representation. To address this critical issue, we take long-range dependencies of input images into account for image style transfer by proposing a transformer-based approach called StyTr2. In contrast with visual transformers for other vision tasks, StyTr2 contains two different transformer encoders to generate domain-specific sequences for content and style, respectively. Following the encoders, a multi-layer transformer decoder is adopted to stylize the content sequence according to the style sequence. We also analyze the deficiency of existing positional encoding methods and propose the content-aware positional encoding (CAPE), which is scale-invariant and more suitable for image style transfer tasks. Qualitative and quantitative experiments demonstrate the effectiveness of the proposed StyTr2 compared with state-of-the-art CNN-based and flow-based approaches. Code and models are available at https://github.com/diyiiyiii/StyTR-2.