# Nested Attention: Semantic-aware Attention Values for Concept Personalization

arXiv: https://arxiv.org/abs/2501.01407 · HTML: https://mlsanity.com/paper/2501.01407/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 70 references that are in the map.
- First posted on arXiv 2 Jan 2025.

## Builds on (most cited first, 30 of 55)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [Zero-shot Image-to-Image Translation](https://mlsanity.com/paper/2302.03027/) (2023)
- [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023)
- [TokenFlow: Consistent Diffusion Features for Consistent Video Editing](https://mlsanity.com/paper/2307.10373/) (2023)
- [ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation](https://mlsanity.com/paper/2302.13848/) (2023)
- [InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning](https://mlsanity.com/paper/2304.03411/) (2023)
- [OminiControl: Minimal and Universal Control for Diffusion Transformer](https://mlsanity.com/paper/2411.15098/) (2024)
- [SVDiff: Compact Parameter Space for Diffusion Fine-Tuning](https://mlsanity.com/paper/2303.11305/) (2023)
- [PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding](https://mlsanity.com/paper/2312.04461/) (2023)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [P+: Extended Textual Conditioning in Text-to-Image Generation](https://mlsanity.com/paper/2303.09522/) (2023)
- [Break-A-Scene: Extracting Multiple Concepts from a Single Image](https://mlsanity.com/paper/2305.16311/) (2023)
- [Subject-driven Text-to-Image Generation via Apprenticeship Learning](https://mlsanity.com/paper/2304.00186/) (2023)
- [Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models](https://mlsanity.com/paper/2302.12228/) (2023)

## Built on by (most cited first, 21 of 21)

- [Personalize Anything for Free with Diffusion Transformer](https://mlsanity.com/paper/2503.12590/) (2025)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [WithAnyone: Towards Controllable and ID Consistent Image Generation](https://mlsanity.com/paper/2510.14975/) (2025)
- [Be Decisive: Noise-Induced Layouts for Multi-Subject Generation](https://mlsanity.com/paper/2505.21488/) (2025)
- [Tight Inversion: Image-Conditioned Inversion for Real Image Editing](https://mlsanity.com/paper/2502.20376/) (2025)
- [ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generation](https://mlsanity.com/paper/2509.18092/) (2025)
- [Piece it Together: Part-Based Concepting with IP-Priors](https://mlsanity.com/paper/2503.10365/) (2025)
- [Canvas-to-Image: Compositional Image Generation with Multimodal Controls](https://mlsanity.com/paper/2511.21691/) (2025)
- [CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvas](https://mlsanity.com/paper/2510.20820/) (2025)
- [Tuning-free Visual Effect Transfer across Videos](https://mlsanity.com/paper/2601.07833/) (2026)
- [DSH-Bench: A Difficulty- and Scenario-Aware Benchmark with Hierarchical Subject Taxonomy for Subject-Driven Text-to-Image Generation](https://mlsanity.com/paper/2603.08090/) (2026)
- [Griffin: Generative Reference and Layout Guided Image Composition](https://mlsanity.com/paper/2509.23643/) (2025)
- [SpotDiff: Spotting and Disentangling Interference in Feature Space for Subject-Preserving Image Generation](https://mlsanity.com/paper/2510.07340/) (2025)
- [Dominant vs. Dominated: Concept-Level Generative Collapse in Diffusion Models](https://mlsanity.com/paper/2512.20666/) (2025)
- [Single Image Iterative Subject-driven Generation and Editing](https://mlsanity.com/paper/2503.16025/) (2025)
- [Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalization](https://mlsanity.com/paper/2512.03964/) (2025)
- [Muses: Designing, Composing, Generating Nonexistent Fantasy 3D Creatures without Training](https://mlsanity.com/paper/2601.03256/) (2026)
- [GimmBO: Interactive Generative Image Model Merging via Bayesian Optimization](https://mlsanity.com/paper/2601.18585/) (2026)
- [Disentangling to Re-couple: Resolving the Similarity-Controllability Paradox in Subject-Driven Text-to-Image Generation](https://mlsanity.com/paper/2604.00849/) (2026)
- [Latent-Identity Tuning in Text-to-Image Personalization Models](https://mlsanity.com/paper/2607.11885/) (2026)
- [Attention Manifolds: Steering or Blocking Language Models by Editing Learned B-Spline Surfaces](https://mlsanity.com/paper/2610.00257/) (2026)

## Abstract (arXiv)

Personalizing text-to-image models to generate images of specific subjects across diverse scenes and styles is a rapidly advancing field. Current approaches often face challenges in maintaining a balance between identity preservation and alignment with the input text prompt. Some methods rely on a single textual token to represent a subject, which limits expressiveness, while others employ richer representations but disrupt the model's prior, diminishing prompt alignment. In this work, we introduce Nested Attention, a novel mechanism that injects a rich and expressive image representation into the model's existing cross-attention layers. Our key idea is to generate query-dependent subject values, derived from nested attention layers that learn to select relevant subject features for each region in the generated image. We integrate these nested layers into an encoder-based personalization method, and show that they enable high identity preservation while adhering to input text prompts. Our approach is general and can be trained on various domains. Additionally, its prior preservation allows us to combine multiple personalized subjects from different domains in a single image.