# Attention Distillation: A Unified Approach to Visual Characteristics Transfer

arXiv: https://arxiv.org/abs/2502.20235 · HTML: https://mlsanity.com/paper/2502.20235/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 48 of its 73 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 48)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [Style Aligned Image Generation via Shared Attention](https://mlsanity.com/paper/2312.02133/) (2023)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022)
- [Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer](https://mlsanity.com/paper/2312.09008/) (2023)
- [StyTr2: Image Style Transfer with Transformers](https://mlsanity.com/paper/2105.14576/) (2021)

## Built on by (most cited first, 30 of 33)

- [REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training](https://mlsanity.com/paper/2505.16792/) (2025)
- [Edit Transfer: Learning Image Editing via Vision In-Context Relations](https://mlsanity.com/paper/2503.13327/) (2025)
- [LookWhere? Efficient Visual Recognition by Learning Where to Look and What to See from Self-Supervision](https://mlsanity.com/paper/2505.18051/) (2025)
- [MegaStyle: Constructing Diverse and Scalable Style Dataset via Consistent Text-to-Image Style Mapping](https://mlsanity.com/paper/2604.08364/) (2026)
- [Attention in Diffusion Model: A Survey](https://mlsanity.com/paper/2504.03738/) (2025)
- [Muon-Accelerated Attention Distillation for Real-Time Edge Synthesis via Optimized Latent Diffusion](https://mlsanity.com/paper/2504.08451/) (2025)
- [StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References](https://mlsanity.com/paper/2603.10354/) (2026)
- [Dynamic Importance in Diffusion U-Net for Enhanced Image Synthesis](https://mlsanity.com/paper/2504.03471/) (2025)
- [From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs](https://mlsanity.com/paper/2504.13471/) (2025)
- [OmniStyle2: Learning to Stylize by Learning to Destylize](https://mlsanity.com/paper/2509.05970/) (2025)
- [Mixture of Style Experts for Diverse Image Stylization](https://mlsanity.com/paper/2603.16649/) (2026)
- [EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing](https://mlsanity.com/paper/2605.07455/) (2026)
- [Cut-ViT: Task-Specific Model Pruning via Gram Anchoring Subspace Consistency](https://mlsanity.com/paper/2608.28205/) (2026)
- [SSIMBaD: Sigma Scaling with SSIM-Guided Balanced Diffusion for AnimeFace Colorization](https://mlsanity.com/paper/2506.04283/) (2025)
- [SPG: Style-Prompting Guidance for Style-Specific Content Creation](https://mlsanity.com/paper/2508.11476/) (2025)
- [Visually Grounded Narratives: Reducing Cognitive Burden in Researcher-Participant Interaction](https://mlsanity.com/paper/2509.00381/) (2025)
- [One-shot Embroidery Customization via Contrastive LoRA Modulation](https://mlsanity.com/paper/2509.18948/) (2025)
- [Image Generation Based on Image Style Extraction](https://mlsanity.com/paper/2510.01347/) (2025)
- [Text to Sketch Generation with Multi-Styles](https://mlsanity.com/paper/2511.04123/) (2025)
- [V-Shuffle: Zero-Shot Style Transfer via Value Shuffle](https://mlsanity.com/paper/2511.06365/) (2025)
- [LumiX: Structured and Coherent Text-to-Intrinsic Generation](https://mlsanity.com/paper/2512.02781/) (2025)
- [DiffStyle3D: Consistent 3D Gaussian Stylization via Attention Optimization](https://mlsanity.com/paper/2601.19717/) (2026)
- [DICE: Disentangling Artist Style from Content via Contrastive Subspace Decomposition in Diffusion Models](https://mlsanity.com/paper/2602.08059/) (2026)
- [HAM: A Training-Free Style Transfer Approach via Heterogeneous Attention Modulation for Diffusion Models](https://mlsanity.com/paper/2603.24043/) (2026)
- [Abstraction in Style: Beyond Texture and Color](https://mlsanity.com/paper/2603.29924/) (2026)
- [Structured 3D Latents Are Surprisingly Powerful: Unleashing Generalizable Style with 2D Diffusion](https://mlsanity.com/paper/2605.04412/) (2026)
- [Semantic-Structural Alignment for Generative Pictorial Charts](https://mlsanity.com/paper/2606.06498/) (2026)
- [Capacity-Controlled Multi-View Stylization of 3D Gaussian Splatting](https://mlsanity.com/paper/2606.26754/) (2026)
- [PoseAlign: Sculpting Pose-Consistent Meshes via Text-Guided Deformation](https://mlsanity.com/paper/2607.10560/) (2026)
- [DReSG: Diffusion Residuals for Stylized Gaussian Splatting](https://mlsanity.com/paper/2608.29048/) (2026)

## Abstract (arXiv)

Recent advances in generative diffusion models have shown a notable inherent understanding of image style and semantics. In this paper, we leverage the self-attention features from pretrained diffusion networks to transfer the visual characteristics from a reference to generated images. Unlike previous work that uses these features as plug-and-play attributes, we propose a novel attention distillation loss calculated between the ideal and current stylization results, based on which we optimize the synthesized image via backpropagation in latent space. Next, we propose an improved Classifier Guidance that integrates attention distillation loss into the denoising sampling process, further accelerating the synthesis and enabling a broad range of image generation applications. Extensive experiments have demonstrated the extraordinary performance of our approach in transferring the examples' style, appearance, and texture to new images in synthesis. Code is available at https://github.com/xugao97/AttentionDistillation.