mlsanityData to 14 Oct 2026

InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Framework

arXiv 2504.12395 ↗First posted 16 Apr 2025cs.CVcited by 19

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Current learning-based subject customization approaches, predominantly relying on U-Net architectures, suffer from limited generalization ability and compromised image quality. Meanwhile, optimization-based methods require subject-specific fine-tuning, which inevitably degrades textual controllability. To address these challenges, we propose InstantCharacter, a scalable framework for character customization built upon a foundation diffusion transformer. InstantCharacter demonstrates three fundamental advantages: first, it achieves open-domain personalization across diverse character appearances, poses, and styles while maintaining high-fidelity results. Second, the framework introduces a scalable adapter with stacked transformer encoders, which effectively processes open-domain character features and seamlessly interacts with the latent space of modern diffusion transformers. Third, to effectively train the framework, we construct a large-scale character dataset containing 10-million-level samples. The dataset is systematically organized into paired (multi-view character) and unpaired (text-image combinations) subsets. This dual-data structure enables simultaneous optimization of identity consistency and textual editability through distinct learning pathways. Qualitative experiments demonstrate the advanced capabilities of InstantCharacter in generating high-fidelity, text-controllable, and character-consistent images, setting a new benchmark for character-driven image generation. Our source code is available at https://github.com/Tencent/InstantCharacter.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 25 of its 25 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

25 of 25 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Personalize Anything for Free with Diffusion Transformer16 Mar
  2. 2024CharacterFactory: Sampling Consistent Characters with GANs for Diffusion Models24 Apr
  3. 2024MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation8 Apr
  4. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
  5. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  6. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  7. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  8. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  7. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  9. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
  10. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  11. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  12. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
4 more
  1. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
  2. 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
  3. 2024RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customizationcited by 39
  4. 2024RealCustom++: Representing Images as Real Textual Word for Real-Time Customizationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 11.

19 papers.

Sort:

By area:

  1. 2025UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Modelscited by 39
  2. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 32
  3. 2025DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editingcited by 29
  4. 2025PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewardscited by 9
  5. 2026Re-Align: Structured Reasoning-guided Alignment for In-Context Image Generation and Editingcited by 8
  6. 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 8
  7. 2025RepText: Rendering Visual Text via Replicatingcited by 7
  8. 2026Omni-3DEdit: Generalized Versatile 3D Editing in One-Passcited by 6
  9. 2026VOSR: A Vision-Only Generative Model for Image Super-Resolutioncited by 5
  10. 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
  11. 2025PersonaVlog: Personalized Multimodal Vlog Generation with Multi-Agent Collaboration and Iterative Self-Correctioncited by 4
  12. 2025Beyond the Pixels: VLM-based Evaluation of Identity Preservation in Reference-Guided Synthesiscited by 4
7 more
  1. 2025AgeBooth: Controllable Facial Aging and Rejuvenation via Diffusion Modelscited by 3
  2. 2026HY-WU (Part I): An Extensible Functional Neural Memory Framework and An Instantiation in Text-Guided Image Editingcited by 3
  3. 2026BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformerscited by 1
  4. 2026Beyond Facial Consistency: Personalized Person Image Generation with Holistic Identity Preservationcited by 1
  5. 2026StreamWise: Serving Multi-Modal Generation in Real-Time at Scalecited by 0
  6. 2026AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvascited by 0
  7. 2026Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Zero-Shot Personalization of Objects via Textual Inversion
  2. 2024LCM-Lookahead for Encoder-based Text-to-Image Personalization
  3. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning
  4. 2026CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets
  5. 2025FreeCus: Free Lunch Subject-driven Customization in Diffusion Transformers
  6. 2024Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion Models
  7. 2023Highly Personalized Text Embedding for Image Manipulation by Stable Diffusion
  8. 2024IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models