# ITstyler: Image-optimized Text-based Style Transfer

arXiv: https://arxiv.org/abs/2301.10916 · HTML: https://mlsanity.com/paper/2301.10916/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 26 Jan 2023.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [StyTr2: Image Style Transfer with Transformers](https://mlsanity.com/paper/2105.14576/) (2021)
- [AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer](https://mlsanity.com/paper/2108.03647/) (2021)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [Dynamic Instance Normalization for Arbitrary Style Transfer](https://mlsanity.com/paper/1911.06953/) (2019)
- [HairCLIP: Design Your Hair by Text and Reference Image](https://mlsanity.com/paper/2112.05142/) (2021)
- [Recognizing Image Style](https://mlsanity.com/paper/1311.3715/) (2013)
- [Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes](https://mlsanity.com/paper/2103.17185/) (2021)
- [Language-Driven Image Style Transfer](https://mlsanity.com/paper/2106.00178/) (2021)

## Built on by (most cited first, 3 of 3)

- [ControlStyle: Text-Driven Stylized Image Generation Using Diffusion Priors](https://mlsanity.com/paper/2311.05463/) (2023)
- [MOSAIC: Multi-Object Segmented Arbitrary Stylization Using CLIP](https://mlsanity.com/paper/2309.13716/) (2023)
- [LEAST: "Local" text-conditioned image style transfer](https://mlsanity.com/paper/2405.16330/) (2024)

## Abstract (arXiv)

Text-based style transfer is a newly-emerging research topic that uses text information instead of style image to guide the transfer process, significantly extending the application scenario of style transfer. However, previous methods require extra time for optimization or text-image paired data, leading to limited effectiveness. In this work, we achieve a data-efficient text-based style transfer method that does not require optimization at the inference stage. Specifically, we convert text input to the style space of the pre-trained VGG network to realize a more effective style swap. We also leverage CLIP's multi-modal embedding space to learn the text-to-style mapping with the image dataset only. Our method can transfer arbitrary new styles of text input in real-time and synthesize high-quality artistic images.