# CoRe: Context-Regularized Text Embedding Learning for Text-to-Image Personalization

arXiv: https://arxiv.org/abs/2408.15914 · HTML: https://mlsanity.com/paper/2408.15914/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 28 Aug 2024.

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning](https://mlsanity.com/paper/2304.03411/) (2023)
- [PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding](https://mlsanity.com/paper/2312.04461/) (2023)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [Break-A-Scene: Extracting Multiple Concepts from a Single Image](https://mlsanity.com/paper/2305.16311/) (2023)
- [Controlling Text-to-Image Diffusion by Orthogonal Finetuning](https://mlsanity.com/paper/2306.07280/) (2023)
- [Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models](https://mlsanity.com/paper/2302.12228/) (2023)
- [Key-Locked Rank One Editing for Text-to-Image Personalization](https://mlsanity.com/paper/2305.01644/) (2023)
- [Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning](https://mlsanity.com/paper/2307.11410/) (2023)
- [DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization](https://mlsanity.com/paper/2402.09812/) (2024)
- [Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approach](https://mlsanity.com/paper/2305.13579/) (2023)
- [RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization](https://mlsanity.com/paper/2403.00483/) (2024)
- [PALP: Prompt Aligned Personalization of Text-to-Image Models](https://mlsanity.com/paper/2401.06105/) (2024)
- [StableIdentity: Inserting Anybody into Anywhere at First Sight](https://mlsanity.com/paper/2401.15975/) (2024)
- [Compositional Inversion for Stable Diffusion Models](https://mlsanity.com/paper/2312.08048/) (2023)
- [ClassDiffusion: More Aligned Personalization Tuning with Explicit Class Guidance](https://mlsanity.com/paper/2405.17532/) (2024)
- [Face2Diffusion for Fast and Editable Face Personalization](https://mlsanity.com/paper/2403.05094/) (2024)

## Built on by (most cited first, 14 of 14)

- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [T-LoRA: Single Image Diffusion Model Customization Without Overfitting](https://mlsanity.com/paper/2507.05964/) (2025)
- [MIGE: Mutually Enhanced Multimodal Instruction-Based Image Generation and Editing](https://mlsanity.com/paper/2502.21291/) (2025)
- [Text-to-Image Synthesis: A Decade Survey](https://mlsanity.com/paper/2411.16164/) (2024)
- [DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition](https://mlsanity.com/paper/2504.17349/) (2025)
- [Detail++: Training-Free Detail Enhancer for T2I Diffusion Models](https://mlsanity.com/paper/2507.17853/) (2025)
- [Low-Rank Continual Personalization of Diffusion Models](https://mlsanity.com/paper/2410.04891/) (2024)
- [Directional Textual Inversion for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2512.13672/) (2025)
- [ACCORD: Alleviating Concept Coupling through Dependence Regularization for Text-to-Image Diffusion Personalization](https://mlsanity.com/paper/2503.01122/) (2025)
- [Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization](https://mlsanity.com/paper/2606.04797/) (2026)
- [Decoupled Guidance: Disentangling Subject and Context Pathways in Text-to-Image Personalization](https://mlsanity.com/paper/2607.00766/) (2026)
- [SEAL: Semantic-aware Single-image Sticker Personalization with a Large-scale Sticker-tag Dataset](https://mlsanity.com/paper/2604.26883/) (2026)
- [STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model](https://mlsanity.com/paper/2606.10653/) (2026)
- [V-Engram: Trigger-Indexed External Memory for Modular Text-to-Image Personalization](https://mlsanity.com/paper/2609.37198/) (2026)

## Abstract (arXiv)

Recent advances in text-to-image personalization have enabled high-quality and controllable image synthesis for user-provided concepts. However, existing methods still struggle to balance identity preservation with text alignment. Our approach is based on the fact that generating prompt-aligned images requires a precise semantic understanding of the prompt, which involves accurately processing the interactions between the new concept and its surrounding context tokens within the CLIP text encoder. To address this, we aim to embed the new concept properly into the input embedding space of the text encoder, allowing for seamless integration with existing tokens. We introduce Context Regularization (CoRe), which enhances the learning of the new concept's text embedding by regularizing its context tokens in the prompt. This is based on the insight that appropriate output vectors of the text encoder for the context tokens can only be achieved if the new concept's text embedding is correctly learned. CoRe can be applied to arbitrary prompts without requiring the generation of corresponding images, thus improving the generalization of the learned text embedding. Additionally, CoRe can serve as a test-time optimization technique to further enhance the generations for specific prompts. Comprehensive experiments demonstrate that our method outperforms several baseline methods in both identity preservation and text alignment. Code will be made publicly available.