# Z^*: Zero-shot Style Transfer via Attention Rearrangement

arXiv: https://arxiv.org/abs/2311.16491 · HTML: https://mlsanity.com/paper/2311.16491/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 42 of its 49 references that are in the map.
- First posted on arXiv 25 Nov 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [Arbitrary Style Transfer with Style-Attentional Networks](https://mlsanity.com/paper/1812.02342/) (2018)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [StyTr2: Image Style Transfer with Transformers](https://mlsanity.com/paper/2105.14576/) (2021)
- [SINE: SINgle Image Editing with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.04489/) (2022)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows](https://mlsanity.com/paper/2103.16877/) (2021)
- [Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning](https://mlsanity.com/paper/2205.09542/) (2022)
- [Arbitrary Video Style Transfer via Multi-Channel Correlation](https://mlsanity.com/paper/2009.08003/) (2020)
- [Effective Real Image Editing with Accelerated Iterative Diffusion Inversion](https://mlsanity.com/paper/2309.04907/) (2023)
- [Arbitrary Style Transfer via Multi-Adaptation Network](https://mlsanity.com/paper/2005.13219/) (2020)

## Built on by (most cited first, 17 of 17)

- [FreeTuner: Any Subject in Any Style with Training-free Diffusion](https://mlsanity.com/paper/2405.14201/) (2024)
- [Multi-turn Consistent Image Editing](https://mlsanity.com/paper/2505.04320/) (2025)
- [UniVST: A Unified Framework for Training-free Localized Video Style Transfer](https://mlsanity.com/paper/2410.20084/) (2024)
- [Unified Diffusion-Based Rigid and Non-Rigid Editing with Text and Image Guidance](https://mlsanity.com/paper/2401.02126/) (2024)
- [An Efficient and Harmonized Framework for Balanced Cross-Domain Feature Integration](https://mlsanity.com/paper/2403.18461/) (2024)
- [Cora: Correspondence-aware image editing using few step diffusion](https://mlsanity.com/paper/2505.23907/) (2025)
- [Attention in Diffusion Model: A Survey](https://mlsanity.com/paper/2504.03738/) (2025)
- [Domain Generalizable Portrait Style Transfer](https://mlsanity.com/paper/2507.04243/) (2025)
- [OmniText: A Training-Free Generalist for Controllable Text-Image Manipulation](https://mlsanity.com/paper/2510.24093/) (2025)
- [ZePo: Zero-Shot Portrait Stylization with Faster Sampling](https://mlsanity.com/paper/2408.05492/) (2024)
- [StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References](https://mlsanity.com/paper/2603.10354/) (2026)
- [Semantix: An Energy Guided Sampler for Semantic Style Transfer](https://mlsanity.com/paper/2503.22344/) (2025)
- [Griffin: Generative Reference and Layout Guided Image Composition](https://mlsanity.com/paper/2509.23643/) (2025)
- [Styleclone: Face Stylization with Diffusion Based Data Augmentation](https://mlsanity.com/paper/2508.17045/) (2025)
- [One-shot Embroidery Customization via Contrastive LoRA Modulation](https://mlsanity.com/paper/2509.18948/) (2025)
- [ASTAD: Asymmetric Style Transfer for Synthetic-to-Real Adaptation in Autonomous Driving](https://mlsanity.com/paper/2606.29286/) (2026)
- [Motion4Motion: Motion Transfer Across Subjects at Inference](https://mlsanity.com/paper/2607.11644/) (2026)

## Abstract (arXiv)

Despite the remarkable progress in image style transfer, formulating style in the context of art is inherently subjective and challenging. In contrast to existing learning/tuning methods, this study shows that vanilla diffusion models can directly extract style information and seamlessly integrate the generative prior into the content image without retraining. Specifically, we adopt dual denoising paths to represent content/style references in latent space and then guide the content image denoising process with style latent codes. We further reveal that the cross-attention mechanism in latent diffusion models tends to blend the content and style images, resulting in stylized outputs that deviate from the original content image. To overcome this limitation, we introduce a cross-attention rearrangement strategy. Through theoretical analysis and experiments, we demonstrate the effectiveness and superiority of the diffusion-based underlineZero-shot underlineStyle underlineTransfer via underlineAttention underlineRearrangement, Z-STAR.