# Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer

arXiv: https://arxiv.org/abs/2312.09008 · HTML: https://mlsanity.com/paper/2312.09008/

## Key facts

- Cited by 196 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 45 of its 55 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing](https://mlsanity.com/paper/2306.14435/) (2023)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation](https://mlsanity.com/paper/2306.07954/) (2023)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [StyTr2: Image Style Transfer with Transformers](https://mlsanity.com/paper/2105.14576/) (2021)
- [StableVideo: Text-driven Consistency-aware Diffusion Video Editing](https://mlsanity.com/paper/2308.09592/) (2023)

## Built on by (most cited first, 30 of 196)

- [CSGO: Content-Style Composition in Text-to-Image Generation](https://mlsanity.com/paper/2408.16766/) (2024)
- [StyleShot: A Snapshot on Any Style](https://mlsanity.com/paper/2407.01414/) (2024)
- [Insert Anything: Image Insertion via In-Context Editing in DiT](https://mlsanity.com/paper/2504.15009/) (2025)
- [InstantStyle-Plus: Style Transfer with Content-Preserving in Text-to-Image Generation](https://mlsanity.com/paper/2407.00788/) (2024)
- [K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs](https://mlsanity.com/paper/2502.18461/) (2025)
- [StyleMaster: Stylize Your Video with Artistic Generation and Translation](https://mlsanity.com/paper/2412.07744/) (2024)
- [AeroGen: Enhancing Remote Sensing Object Detection with Diffusion-Driven Data Generation](https://mlsanity.com/paper/2411.15497/) (2024)
- [Attention Distillation: A Unified Approach to Visual Characteristics Transfer](https://mlsanity.com/paper/2502.20235/) (2025)
- [OmniStyle: Filtering High Quality Style Transfer Data at Scale](https://mlsanity.com/paper/2505.14028/) (2025)
- [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024)
- [USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learning](https://mlsanity.com/paper/2508.18966/) (2025)
- [StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elements](https://mlsanity.com/paper/2412.08503/) (2024)
- [FreeTuner: Any Subject in Any Style with Training-free Diffusion](https://mlsanity.com/paper/2405.14201/) (2024)
- [StyleSSP: Sampling StartPoint Enhancement for Training-free Diffusion-based Method for Style Transfer](https://mlsanity.com/paper/2501.11319/) (2025)
- [AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising](https://mlsanity.com/paper/2406.06911/) (2024)
- [Multi-turn Consistent Image Editing](https://mlsanity.com/paper/2505.04320/) (2025)
- [ViPer: Visual Personalization of Generative Models via Individual Preference Learning](https://mlsanity.com/paper/2407.17365/) (2024)
- [UniVST: A Unified Framework for Training-free Localized Video Style Transfer](https://mlsanity.com/paper/2410.20084/) (2024)
- [DiffV2IR: Visible-to-Infrared Diffusion Model via Vision-Language Understanding](https://mlsanity.com/paper/2503.19012/) (2025)
- [ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation](https://mlsanity.com/paper/2509.24878/) (2025)
- [Personalized Vision via Visual In-Context Learning](https://mlsanity.com/paper/2509.25172/) (2025)
- [Fine-Tuning Visual Autoregressive Models for Subject-Driven Generation](https://mlsanity.com/paper/2504.02612/) (2025)
- [U-StyDiT: Ultra-high Quality Artistic Style Transfer Using Diffusion Transformers](https://mlsanity.com/paper/2503.08157/) (2025)
- [Color Alignment in Diffusion](https://mlsanity.com/paper/2503.06746/) (2025)
- [O-DisCo-Edit: Object Distortion Control for Unified Realistic Video Editing](https://mlsanity.com/paper/2509.01596/) (2025)
- [AttenST: A Training-Free Attention-Driven Style Transfer Framework with Pre-Trained Diffusion Models](https://mlsanity.com/paper/2503.07307/) (2025)
- [ConsisLoRA: Enhancing Content and Style Consistency for LoRA-based Style Transfer](https://mlsanity.com/paper/2503.10614/) (2025)
- [Free-Lunch Color-Texture Disentanglement for Stylized Image Generation](https://mlsanity.com/paper/2503.14275/) (2025)
- [SA-LUT: Spatial Adaptive 4D Look-Up Table for Photorealistic Style Transfer](https://mlsanity.com/paper/2506.13465/) (2025)
- [An Efficient and Harmonized Framework for Balanced Cross-Domain Feature Integration](https://mlsanity.com/paper/2403.18461/) (2024)

## Abstract (arXiv)

Despite the impressive generative capabilities of diffusion models, existing diffusion model-based style transfer methods require inference-stage optimization (e.g. fine-tuning or textual inversion of style) which is time-consuming, or fails to leverage the generative ability of large-scale diffusion models. To address these issues, we introduce a novel artistic style transfer method based on a pre-trained large-scale diffusion model without any optimization. Specifically, we manipulate the features of self-attention layers as the way the cross-attention mechanism works; in the generation process, substituting the key and value of content with those of style image. This approach provides several desirable characteristics for style transfer including 1) preservation of content by transferring similar styles into similar image patches and 2) transfer of style based on similarity of local texture (e.g. edge) between content and style images. Furthermore, we introduce query preservation and attention temperature scaling to mitigate the issue of disruption of original content, and initial latent Adaptive Instance Normalization (AdaIN) to deal with the disharmonious color (failure to transfer the colors of style). Our experimental results demonstrate that our proposed method surpasses state-of-the-art methods in both conventional and diffusion-based style transfer baselines.