# DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization

arXiv: https://arxiv.org/abs/2402.09812 · HTML: https://mlsanity.com/paper/2402.09812/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 60 of its 70 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 60)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation](https://mlsanity.com/paper/2302.13848/) (2023)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning](https://mlsanity.com/paper/2304.03411/) (2023)
- [SVDiff: Compact Parameter Space for Diffusion Fine-Tuning](https://mlsanity.com/paper/2303.11305/) (2023)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023)

## Built on by (most cited first, 30 of 60)

- [Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance](https://mlsanity.com/paper/2403.17377/) (2024)
- [Attentive Eraser: Unleashing Diffusion Model's Object Removal Potential via Self-Attention Redirection Guidance](https://mlsanity.com/paper/2412.12974/) (2024)
- [CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities](https://mlsanity.com/paper/2408.13239/) (2024)
- [ControlVAR: Exploring Controllable Visual Autoregressive Modeling](https://mlsanity.com/paper/2406.09750/) (2024)
- [Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignment](https://mlsanity.com/paper/2403.12965/) (2024)
- [EditWorld: Simulating World Dynamics for Instruction-Following Image Editing](https://mlsanity.com/paper/2405.14785/) (2024)
- [Emergent Temporal Correspondences from Video Diffusion Transformers](https://mlsanity.com/paper/2506.17220/) (2025)
- [MC2: Multi-concept Guidance for Customized Multi-concept Generation](https://mlsanity.com/paper/2404.05268/) (2024)
- [StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elements](https://mlsanity.com/paper/2412.08503/) (2024)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [Multi-turn Consistent Image Editing](https://mlsanity.com/paper/2505.04320/) (2025)
- [Diffusion-Based Visual Art Creation: A Survey and New Perspectives](https://mlsanity.com/paper/2408.12128/) (2024)
- [Exploring Multimodal Diffusion Transformers for Enhanced Prompt-based Image Editing](https://mlsanity.com/paper/2508.07519/) (2025)
- [RealCustom++: Representing Images as Real Textual Word for Real-Time Customization](https://mlsanity.com/paper/2408.09744/) (2024)
- [T-LoRA: Single Image Diffusion Model Customization Without Overfitting](https://mlsanity.com/paper/2507.05964/) (2025)
- [CoRe: Context-Regularized Text Embedding Learning for Text-to-Image Personalization](https://mlsanity.com/paper/2408.15914/) (2024)
- [CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customization](https://mlsanity.com/paper/2409.05606/) (2024)
- [Fine-Tuning Visual Autoregressive Models for Subject-Driven Generation](https://mlsanity.com/paper/2504.02612/) (2025)
- [AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation](https://mlsanity.com/paper/2406.05000/) (2024)
- [Consistent Image Layout Editing with Diffusion Models](https://mlsanity.com/paper/2503.06419/) (2025)
- [Dual-Schedule Inversion: Training- and Tuning-Free Inversion for Real Image Editing](https://mlsanity.com/paper/2412.11152/) (2024)
- [Incorporating Visual Correspondence into Diffusion Model for Virtual Try-On](https://mlsanity.com/paper/2505.16977/) (2025)
- [Visual Persona: Foundation Model for Full-Body Human Customization](https://mlsanity.com/paper/2503.15406/) (2025)
- [Steering Guidance for Personalized Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.00319/) (2025)
- [DissolveStereo: Coarse Depth Injection for Zero-Shot Stereo Video Generation](https://mlsanity.com/paper/2411.14295/) (2024)
- [Where and How to Perturb: On the Design of Perturbation Guidance in Diffusion and Flow Models](https://mlsanity.com/paper/2506.10978/) (2025)
- [DreamSteerer: Enhancing Source Image Conditioned Editability using Personalized Diffusion Models](https://mlsanity.com/paper/2410.11208/) (2024)
- [FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization](https://mlsanity.com/paper/2507.01792/) (2025)
- [CoreEditor: Correspondence-constrained Diffusion for Consistent 3D Editing](https://mlsanity.com/paper/2508.11603/) (2025)
- [Attention in Diffusion Model: A Survey](https://mlsanity.com/paper/2504.03738/) (2025)

## Abstract (arXiv)

The objective of text-to-image (T2I) personalization is to customize a diffusion model to a user-provided reference concept, generating diverse images of the concept aligned with the target prompts. Conventional methods representing the reference concepts using unique text embeddings often fail to accurately mimic the appearance of the reference. To address this, one solution may be explicitly conditioning the reference images into the target denoising process, known as key-value replacement. However, prior works are constrained to local editing since they disrupt the structure path of the pre-trained T2I model. To overcome this, we propose a novel plug-in method, called DreamMatcher, which reformulates T2I personalization as semantic matching. Specifically, DreamMatcher replaces the target values with reference values aligned by semantic matching, while leaving the structure path unchanged to preserve the versatile capability of pre-trained T2I models for generating diverse structures. We also introduce a semantic-consistent masking strategy to isolate the personalized concept from irrelevant regions introduced by the target prompts. Compatible with existing T2I models, DreamMatcher shows significant improvements in complex scenarios. Intensive analyses demonstrate the effectiveness of our approach.