mlsanityData to 14 Oct 2026

Personalize Anything for Free with Diffusion Transformer

arXiv 2503.12590 ↗First posted 16 Mar 2025cs.CVcited by 23

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Personalized image generation aims to produce images of user-specified concepts while enabling flexible editing. Recent training-free approaches, while exhibit higher computational efficiency than training-based methods, struggle with identity preservation, applicability, and compatibility with diffusion transformers (DiTs). In this paper, we uncover the untapped potential of DiT, where simply replacing denoising tokens with those of a reference subject achieves zero-shot subject reconstruction. This simple yet effective feature injection technique unlocks diverse scenarios, from personalization to image editing. Building upon this observation, we propose Personalize Anything, a training-free framework that achieves personalized image generation in DiT through: 1) timestep-adaptive token replacement that enforces subject consistency via early-stage injection and enhances flexibility through late-stage regularization, and 2) patch perturbation strategies to boost structural diversity. Our method seamlessly supports layout-guided generation, multi-subject personalization, and mask-controlled editing. Evaluations demonstrate state-of-the-art performance in identity preservation and versatility. Our work establishes new insights into DiTs while delivering a practical paradigm for efficient personalization.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 61 of its 68 references that are in the map. First posted on arXiv 16 Mar 2025.

Builds on

61 of 68 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025Object-level Visual Prompts for Compositional Image Generation2 Jan
  2. 2024EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM12 Dec
  3. 2024Taming Rectified Flow for Inversion and Editing7 Nov
  4. 2024Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations14 Oct
  5. 2024pOps: Photo-Inspired Diffusion Operators3 Jun
  6. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding14 May
  7. 2024CharacterFactory: Sampling Consistent Characters with GANs for Diffusion Models24 Apr
  8. 2024MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation8 Apr
  9. 2024LCM-Lookahead for Encoder-based Text-to-Image Personalization4 Apr
  10. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models16 Mar
  11. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space7 Feb
  12. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
11 more
  1. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Models16 Nov
  2. 2023MagiCapture: High-Resolution Multi-Concept Portrait Customization13 Sep
  3. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  4. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning21 Jul
  5. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  6. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
  7. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Scalable Diffusion Models with Transformerscited by 6,054
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  12. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
26 more
  1. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
  2. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  3. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  4. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 247
  5. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  6. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  7. 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 144
  8. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
  9. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
  10. 2024Training-Free Consistent Text-to-Image Generationcited by 125
  11. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 102
  12. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
  13. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
  14. 2024FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Compositioncited by 49
  15. 2024Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generatorcited by 48
  16. 2024MultiBooth: Towards Generating All Your Concepts in an Image from Textcited by 45
  17. 2024ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preservingcited by 43
  18. 2024Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generationcited by 43
  19. 2024FlashFace: Human Image Personalization with High-fidelity Identity Preservationcited by 31
  20. 2024MC2: Multi-concept Guidance for Customized Multi-concept Generationcited by 26
  21. 2024Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Modelscited by 23
  22. 2025Nested Attention: Semantic-aware Attention Values for Concept Personalizationcited by 21
  23. 2024From Parts to Whole: A Unified Reference Framework for Controllable Human Image Generationcited by 17
  24. 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
  25. 2024TweedieMix: Improving Multi-Concept Fusion for Diffusion-based Image/Video Generationcited by 14
  26. 2025IP-Prompter: Training-Free Theme-Specific Image Generation via Dynamic Visual Promptingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 18; 2026: 5.

23 papers.

Sort:

By area:

  1. 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 45
  2. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  3. 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learningcited by 25
  4. 2025DiT360: High-Fidelity Panoramic Image Generation via Hybrid Trainingcited by 24
  5. 2025InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Frameworkcited by 19
  6. 2025MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Modelscited by 17
  7. 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
  8. 2025PairEdit: Learning Semantic Variations for Exemplar-based Image Editingcited by 8
  9. 2025FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalizationcited by 5
  10. 2025PositionIC: Unified Position and Identity Consistency for Image Customizationcited by 4
  11. 2025LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondencecited by 4
  12. 2025MultiCrafter: High-Fidelity Multi-Subject Generation via Disentangled Attention and Identity-Aware Preference Alignmentcited by 4
11 more
  1. 2025V-Warper: Appearance-Consistent Video Diffusion Personalization via Value Warpingcited by 3
  2. 2025Personalize Your Gaussian: Consistent 3D Scene Personalization from a Single Imagecited by 2
  3. 2025The Aging Multiverse: Generating Condition-Aware Facial Aging Tree via Training-Free Diffusioncited by 2
  4. 2025Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformercited by 2
  5. 2026JRM: Joint Reconstruction Model for Multiple Objects without Alignmentcited by 2
  6. 2025RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generationcited by 0
  7. 2025Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalizationcited by 0
  8. 2026OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidancecited by 0
  9. 2026RefGC-SR2: Reference-guided Super-Resolution and Refinement of AI Generated Contentcited by 0
  10. 2026V-Engram: Trigger-Indexed External Memory for Modular Text-to-Image Personalizationcited by 0
  11. 2026ScribbleEdit: A Benchmark for Scribble-Only Image Editingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SwiftPie: Lightning-fast Subject-driven Image Personalization via One step Diffusion
  2. 2024DreamSteerer: Enhancing Source Image Conditioned Editability using Personalized Diffusion Models
  3. 2025Create Anything Anywhere: Layout-Controllable Personalized Diffusion Model for Multiple Subjects
  4. 2025Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models
  5. 2026Zero-Shot Personalization of Objects via Textual Inversion
  6. 2026CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets
  7. 2025CusEnhancer: A Zero-Shot Scene and Controllability Enhancement Method for Photo Customization via ResInversion
  8. 2024Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization