# LEAST: "Local" text-conditioned image style transfer

arXiv: https://arxiv.org/abs/2405.16330 · HTML: https://mlsanity.com/paper/2405.16330/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 31 references that are in the map.
- First posted on arXiv 25 May 2024.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Zero-shot Image-to-Image Translation](https://mlsanity.com/paper/2302.03027/) (2023)
- [Text2LIVE: Text-Driven Layered Image and Video Editing](https://mlsanity.com/paper/2204.02491/) (2022)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [Guiding Instruction-based Image Editing via Multimodal Large Language Models](https://mlsanity.com/paper/2309.17102/) (2023)
- [InstructDiffusion: A Generalist Modeling Interface for Vision Tasks](https://mlsanity.com/paper/2309.03895/) (2023)
- [StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models](https://mlsanity.com/paper/2308.07863/) (2023)
- [Expressive Text-to-Image Generation with Rich Text](https://mlsanity.com/paper/2304.06720/) (2023)
- [Zero-Shot Contrastive Loss for Text-Guided Diffusion Image Style Transfer](https://mlsanity.com/paper/2303.08622/) (2023)
- [Language-Driven Image Style Transfer](https://mlsanity.com/paper/2106.00178/) (2021)
- [Name Your Style: An Arbitrary Artist-aware Image Style Transfer](https://mlsanity.com/paper/2202.13562/) (2022)
- [StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model](https://mlsanity.com/paper/2303.09268/) (2023)
- [ITstyler: Image-optimized Text-based Style Transfer](https://mlsanity.com/paper/2301.10916/) (2023)
- [MOSAIC: Multi-Object Segmented Arbitrary Stylization Using CLIP](https://mlsanity.com/paper/2309.13716/) (2023)
- [Sem-CS: Semantic CLIPStyler for Text-Based Image Style Transfer](https://mlsanity.com/paper/2307.05934/) (2023)

## Abstract (arXiv)

Text-conditioned style transfer enables users to communicate their desired artistic styles through text descriptions, offering a new and expressive means of achieving stylization. In this work, we evaluate the text-conditioned image editing and style transfer techniques on their fine-grained understanding of user prompts for precise "local" style transfer. We find that current methods fail to accomplish localized style transfers effectively, either failing to localize style transfer to certain regions in the image, or distorting the content and structure of the input image. To this end, we develop an end-to-end pipeline for "local" style transfer tailored to align with users' intent. Further, we substantiate the effectiveness of our approach through quantitative and qualitative analysis. The project code is available at: https://github.com/silky1708/local-style-transfer.