mlsanityData to 14 Oct 2026

A Survey on Personalized Content Synthesis with Diffusion Models

arXiv 2405.05538 ↗First posted 9 May 2024cs.CVcited by 29

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Recent advancements in diffusion models have significantly impacted content creation, leading to the emergence of Personalized Content Synthesis (PCS). By utilizing a small set of user-provided examples featuring the same subject, PCS aims to tailor this subject to specific user-defined prompts. Over the past two years, more than 150 methods have been introduced in this area. However, existing surveys primarily focus on text-to-image generation, with few providing up-to-date summaries on PCS. This paper provides a comprehensive survey of PCS, introducing the general frameworks of PCS research, which can be categorized into test-time fine-tuning (TTF) and pre-trained adaptation (PTA) approaches. We analyze the strengths, limitations, and key techniques of these methodologies. Additionally, we explore specialized tasks within the field, such as object, face, and style personalization, while highlighting their unique challenges and innovations. Despite the promising progress, we also discuss ongoing challenges, including overfitting and the trade-off between subject fidelity and text alignment. Through this detailed overview and analysis, we propose future directions to further the development of PCS.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 188 of its 246 references that are in the map. First posted on arXiv 9 May 2024.

Builds on

188 of 246 references.

Closest ancestors · 56

Same direction, newest first

  1. 2024ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning23 Apr
  2. 2024Infusion: Preventing Customized Text-to-Image Diffusion from Overfitting22 Apr
  3. 2024MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generation17 Apr
  4. 2024MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation8 Apr
  5. 2024IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models20 Mar
  6. 2024Tuning-Free Image Customization with Image and Text Guidance19 Mar
  7. 2024Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm18 Mar
  8. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models16 Mar
  9. 2024Face2Diffusion for Fast and Editable Face Personalization8 Mar
  10. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey7 Mar
  11. 2024Visual Concept-driven Image Generation with Text-to-Image Diffusion Model18 Feb
  12. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space7 Feb
38 more
  1. 2024Object-Driven One-Shot Fine-tuning of Text-to-Image Diffusion with Prototypical Embedding28 Jan
  2. 2024BootPIG: Bootstrapping Zero-shot Personalized Image Generation Capabilities in Pretrained Diffusion Models25 Jan
  3. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
  4. 2023Cross Initialization for Personalized Text-to-Image Generation26 Dec
  5. 2023DreamDistribution: Learning Prompt Distribution for Diverse In-distribution Generation21 Dec
  6. 2023Compositional Inversion for Stable Diffusion Models13 Dec
  7. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition12 Dec
  8. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization11 Dec
  9. 2023Personalized Face Inpainting with Diffusion Models by Parallel Visual Attention6 Dec
  10. 2023HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models30 Nov
  11. 2023CatVersion: Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization24 Nov
  12. 2023Lego: Learning to Disentangle and Invert Personalized Concepts Beyond Object Appearance in Text-to-Image Diffusion Models23 Nov
  13. 2023An Image is Worth Multiple Words: Multi-attribute Inversion for Constrained Text-to-Image Synthesis20 Nov
  14. 2023High-fidelity Person-centric Subject-to-Image Synthesis17 Nov
  15. 2023CustomNet: Zero-shot Object Customization with Variable-Viewpoints in Text-to-Image Diffusion Models30 Oct
  16. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation26 Sep
  17. 2023MagiCapture: High-Resolution Multi-Concept Portrait Customization13 Sep
  18. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  19. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning21 Jul
  20. 2023Identity-Preserving Aging of Face Images via Latent Diffusion Models17 Jul
  21. 2023Generate Anything Anywhere in Any Scene29 Jun
  22. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuning12 Jun
  23. 2023ViCo: Plug-and-play Visual Condition for Personalized Text-to-image Generation1 Jun
  24. 2023Inserting Anybody in Diffusion Models via Celeb Basis1 Jun
  25. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  26. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  27. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models25 May
  28. 2023Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models25 May
  29. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention17 May
  30. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA12 Apr
  31. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Models5 Apr
  32. 2023ReVersion: Diffusion-Based Relation Inversion from Images23 Mar
  33. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  34. 2023Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation16 Mar
  35. 2023Highly Personalized Text Embedding for Image Manipulation by Stable Diffusion15 Mar
  36. 2023Text-to-image Diffusion Models in Generative AI: A Survey14 Mar
  37. 2023Cones: Concept Neurons in Diffusion Models for Customized Generation9 Mar
  38. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models23 Feb

50 newest of 56 All 56 (JSON)

Other references · 132

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  9. 2023Segment Anythingcited by 8,234
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  12. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
38 more
  1. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  6. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  7. 2016Improved Techniques for Training GANscited by 2,970
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  9. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  11. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  12. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  13. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  14. 2022Diffusion Models in Vision: A Surveycited by 1,015
  15. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  16. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  17. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  18. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  19. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  20. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 903
  21. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  22. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  23. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  24. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
  25. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  26. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
  27. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 345
  28. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  29. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  30. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  31. 2023Personalize Segment Anything Model with One Shotcited by 250
  32. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  33. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  34. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 231
  35. 2023Style Aligned Image Generation via Shared Attentioncited by 216
  36. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 212
  37. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210
  38. 2023A Survey on Video Diffusion Modelscited by 206

50 most cited of 132 All 132 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 16; 2026: 8.

29 papers.

Sort:

By area:

  1. 2024Erasing Concepts from Text-to-Image Diffusion Models with Few-shot Unlearningcited by 29
  2. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  3. 2025On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devicescited by 15
  4. 2024MagicTailor: Component-Controllable Personalization in Text-to-Image Diffusion Modelscited by 14
  5. 2025PersonaBooth: Personalized Text-to-Motion Generationcited by 14
  6. 2024Slight Corruption in Pre-training Data Makes Better Diffusion Modelscited by 9
  7. 2025TaxaDiffusion: Progressively Trained Diffusion Model for Fine-Grained Species Generationcited by 7
  8. 2025Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generationcited by 6
  9. 2025A Comprehensive Survey on Visual Concept Mining in Text-to-image Diffusion Modelscited by 5
  10. 2025Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Modelscited by 5
  11. 2024Protect-Your-IP: Scalable Source-Tracing and Attribution against Personalized Generationcited by 3
  12. 2025Abductive Logical Rule Induction by Bridging Inductive Logic Programming and Multimodal Large Language Modelscited by 3
17 more
  1. 2026TaxaAdapter: Vision Taxonomy Models are Key to Fine-grained Image Generation over the Tree of Lifecited by 3
  2. 2024Foundation Cures Personalization: Improving Personalized Models' Prompt Consistency via Hidden Foundation Knowledgecited by 2
  3. 2025Meta-LoRA: Meta-Learning LoRA Components for Domain-Aware ID Personalizationcited by 1
  4. 2025T2I-ConBench: Text-to-Image Benchmark for Continual Post-trainingcited by 1
  5. 2025Consistent text-to-image generation via scene de-contextualizationcited by 1
  6. 2026Bridging Brain and Semantics: A Hierarchical Framework for Semantically Enhanced fMRI-to-Video Reconstructioncited by 1
  7. 2025Curiosity-Diffuser: Curiosity Guide Diffusion Models for Reliabilitycited by 0
  8. 2025On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devicescited by 0
  9. 2025Per-Query Visual Concept Learningcited by 0
  10. 2025MONKEY: Masking ON KEY-Value Activation Adapter for Personalizationcited by 0
  11. 2025Salient Concept-Aware Generative Data Augmentationcited by 0
  12. 2026GimmBO: Interactive Generative Image Model Merging via Bayesian Optimizationcited by 0
  13. 2026Hierarchical Codec Diffusion for Video-to-Speech Generationcited by 0
  14. 2026AI-T2I: Aggregating-and-Isolating Cross-Attention to Diffusion Models for Text-to-Image Synthesiscited by 0
  15. 2026UniVerse: A Unified Modulation Framework for Segmentation-Free,Disentangled Multi-Concept Personalizationcited by 0
  16. 2026From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generationcited by 0
  17. 2026Anti-Persona: Disrupting Unauthorized Identity Binding and Recognition in Personalized Vision--Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Preserve and Personalize: Personalized Text-to-Image Diffusion Models without Distributional Drift
  2. 2025Personalized Text-to-Image Generation with Auto-Regressive Models
  3. 2024Learning to Customize Text-to-Image Diffusion In Diverse Context
  4. 2024Fast Personalized Text-to-Image Syntheses With Attention Injection
  5. 2024PaRa: Personalizing Text-to-Image Diffusion via Parameter Rank Reduction
  6. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models
  7. 2025Steering Guidance for Personalized Text-to-Image Diffusion Models
  8. 2024Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion Models