Personalize Anything for Free with Diffusion Transformer
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Personalized image generation aims to produce images of user-specified concepts while enabling flexible editing. Recent training-free approaches, while exhibit higher computational efficiency than training-based methods, struggle with identity preservation, applicability, and compatibility with diffusion transformers (DiTs). In this paper, we uncover the untapped potential of DiT, where simply replacing denoising tokens with those of a reference subject achieves zero-shot subject reconstruction. This simple yet effective feature injection technique unlocks diverse scenarios, from personalization to image editing. Building upon this observation, we propose Personalize Anything, a training-free framework that achieves personalized image generation in DiT through: 1) timestep-adaptive token replacement that enforces subject consistency via early-stage injection and enhances flexibility through late-stage regularization, and 2) patch perturbation strategies to boost structural diversity. Our method seamlessly supports layout-guided generation, multi-subject personalization, and mask-controlled editing. Evaluations demonstrate state-of-the-art performance in identity preservation and versatility. Our work establishes new insights into DiTs while delivering a practical paradigm for efficient personalization.
Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 61 of its 68 references that are in the map. First posted on arXiv 16 Mar 2025.
Builds on
61 of 68 references.
Closest ancestors · 23
Same direction, newest first
- 2025Object-level Visual Prompts for Compositional Image Generation2 Jan
- 2024EasyRef: Omni-Generalized Group Image Reference for Diffusion Models via Multimodal LLM12 Dec
- 2024Taming Rectified Flow for Inversion and Editing7 Nov
- 2024Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations14 Oct
- 2024pOps: Photo-Inspired Diffusion Operators3 Jun
- 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding14 May
- 2024CharacterFactory: Sampling Consistent Characters with GANs for Diffusion Models24 Apr
- 2024MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation8 Apr
- 2024LCM-Lookahead for Encoder-based Text-to-Image Personalization4 Apr
- 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models16 Mar
- 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Space7 Feb
- 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
11 more
- 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Models16 Nov
- 2023MagiCapture: High-Resolution Multi-Concept Portrait Customization13 Sep
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
- 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning21 Jul
- 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
- 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuning20 Mar
- 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
Other references · 38
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
26 more
- 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 247
- 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
- 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
- 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 144
- 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
- 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
- 2024Training-Free Consistent Text-to-Image Generationcited by 125
- 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 102
- 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
- 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
- 2024FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Compositioncited by 49
- 2024Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generatorcited by 48
- 2024MultiBooth: Towards Generating All Your Concepts in an Image from Textcited by 45
- 2024ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preservingcited by 43
- 2024Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generationcited by 43
- 2024FlashFace: Human Image Personalization with High-fidelity Identity Preservationcited by 31
- 2024MC2: Multi-concept Guidance for Customized Multi-concept Generationcited by 26
- 2024Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Modelscited by 23
- 2025Nested Attention: Semantic-aware Attention Values for Concept Personalizationcited by 21
- 2024From Parts to Whole: A Unified Reference Framework for Controllable Human Image Generationcited by 17
- 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
- 2024TweedieMix: Improving Multi-Concept Fusion for Diffusion-based Image/Video Generationcited by 14
- 2025IP-Prompter: Training-Free Theme-Specific Image Generation via Dynamic Visual Promptingcited by 3
Built on by
Citing papers per year: 2025: 18; 2026: 5.
23 papers.
Sort:
By area:
- 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 45
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
- 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learningcited by 25
- 2025DiT360: High-Fidelity Panoramic Image Generation via Hybrid Trainingcited by 24
- 2025InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Frameworkcited by 19
- 2025MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Modelscited by 17
- 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
- 2025PairEdit: Learning Semantic Variations for Exemplar-based Image Editingcited by 8
- 2025FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalizationcited by 5
- 2025PositionIC: Unified Position and Identity Consistency for Image Customizationcited by 4
- 2025LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondencecited by 4
- 2025MultiCrafter: High-Fidelity Multi-Subject Generation via Disentangled Attention and Identity-Aware Preference Alignmentcited by 4
11 more
- 2025V-Warper: Appearance-Consistent Video Diffusion Personalization via Value Warpingcited by 3
- 2025Personalize Your Gaussian: Consistent 3D Scene Personalization from a Single Imagecited by 2
- 2025The Aging Multiverse: Generating Condition-Aware Facial Aging Tree via Training-Free Diffusioncited by 2
- 2025Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformercited by 2
- 2026JRM: Joint Reconstruction Model for Multiple Objects without Alignmentcited by 2
- 2025RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generationcited by 0
- 2025Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalizationcited by 0
- 2026OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidancecited by 0
- 2026RefGC-SR2: Reference-guided Super-Resolution and Refinement of AI Generated Contentcited by 0
- 2026V-Engram: Trigger-Indexed External Memory for Modular Text-to-Image Personalizationcited by 0
- 2026ScribbleEdit: A Benchmark for Scribble-Only Image Editingcited by 0
- 2026ScribbleEdit: A Benchmark for Scribble-Only Image Editing7 Oct 2026
- 2026V-Engram: Trigger-Indexed External Memory for Modular Text-to-Image Personalization29 Sep 2026
- 2026RefGC-SR2: Reference-guided Super-Resolution and Refinement of AI Generated Content13 Jun 2026
- 2026JRM: Joint Reconstruction Model for Multiple Objects without Alignment27 Mar 2026
- 2026OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidance16 Feb 2026
- 2025V-Warper: Appearance-Consistent Video Diffusion Personalization via Value Warping13 Dec 2025
- 2025Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalization3 Dec 2025
- 2025Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformer30 Nov 2025
- 2025DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training13 Oct 2025
- 2025MultiCrafter: High-Fidelity Multi-Subject Generation via Disentangled Attention and Identity-Aware Preference Alignment26 Sep 2025
- 2025LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondence15 Sep 2025
- 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Space26 Aug 2025
11 more
- 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learning11 Aug 2025
- 2025PositionIC: Unified Position and Identity Consistency for Image Customization18 Jul 2025
- 2025RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generation3 Jul 2025
- 2025FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization2 Jul 2025
- 2025The Aging Multiverse: Generating Condition-Aware Facial Aging Tree via Training-Free Diffusion26 Jun 2025
- 2025PairEdit: Learning Semantic Variations for Exemplar-based Image Editing9 Jun 2025
- 2025MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models26 May 2025
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design25 May 2025
- 2025Personalize Your Gaussian: Consistent 3D Scene Personalization from a Single Image20 May 2025
- 2025InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Framework16 Apr 2025
- 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generation3 Apr 2025
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
- 2025InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Frameworkcited by 19
- 2025LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondencecited by 4
- 2025V-Warper: Appearance-Consistent Video Diffusion Personalization via Value Warpingcited by 3
- 2025Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformercited by 2
- 2025RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generationcited by 0
- 2025Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalizationcited by 0
- 2025V-Warper: Appearance-Consistent Video Diffusion Personalization via Value Warping13 Dec 2025
- 2025Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalization3 Dec 2025
- 2025Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformer30 Nov 2025
- 2025LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondence15 Sep 2025
- 2025RichControl: Structure- and Appearance-Rich Training-Free Spatial Control for Text-to-Image Generation3 Jul 2025
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design25 May 2025
- 2025InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Framework16 Apr 2025
- 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 45
- 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learningcited by 25
- 2025DiT360: High-Fidelity Panoramic Image Generation via Hybrid Trainingcited by 24
- 2025MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Modelscited by 17
- 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
- 2025PairEdit: Learning Semantic Variations for Exemplar-based Image Editingcited by 8
- 2025FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalizationcited by 5
- 2025PositionIC: Unified Position and Identity Consistency for Image Customizationcited by 4
- 2025Personalize Your Gaussian: Consistent 3D Scene Personalization from a Single Imagecited by 2
- 2025The Aging Multiverse: Generating Condition-Aware Facial Aging Tree via Training-Free Diffusioncited by 2
- 2026JRM: Joint Reconstruction Model for Multiple Objects without Alignmentcited by 2
- 2026OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidancecited by 0
12 most cited of 15 All 15 (JSON)
- 2026ScribbleEdit: A Benchmark for Scribble-Only Image Editing7 Oct 2026
- 2026V-Engram: Trigger-Indexed External Memory for Modular Text-to-Image Personalization29 Sep 2026
- 2026RefGC-SR2: Reference-guided Super-Resolution and Refinement of AI Generated Content13 Jun 2026
- 2026JRM: Joint Reconstruction Model for Multiple Objects without Alignment27 Mar 2026
- 2026OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidance16 Feb 2026
- 2025DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training13 Oct 2025
- 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Space26 Aug 2025
- 2025X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learning11 Aug 2025
- 2025PositionIC: Unified Position and Identity Consistency for Image Customization18 Jul 2025
- 2025FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization2 Jul 2025
- 2025The Aging Multiverse: Generating Condition-Aware Facial Aging Tree via Training-Free Diffusion26 Jun 2025
- 2025PairEdit: Learning Semantic Variations for Exemplar-based Image Editing9 Jun 2025
12 newest of 15 All 15 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026SwiftPie: Lightning-fast Subject-driven Image Personalization via One step Diffusion
- 2024DreamSteerer: Enhancing Source Image Conditioned Editability using Personalized Diffusion Models
- 2025Create Anything Anywhere: Layout-Controllable Personalized Diffusion Model for Multiple Subjects
- 2025Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models
- 2026Zero-Shot Personalization of Objects via Textual Inversion
- 2026CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets
- 2025CusEnhancer: A Zero-Shot Scene and Controllability Enhancement Method for Photo Customization via ResInversion
- 2024Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization