mlsanityData to 7 Oct 2026

ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning

arXiv 2501.04698 ↗First posted 8 Jan 2025cs.CVcited by 67

On the map: Diffusion models → Video generation

Abstract (arXiv)

Text-to-video generation has made remarkable advancements through diffusion models. However, Multi-Concept Video Customization (MCVC) remains a significant challenge. We identify two key challenges for this task: 1) the identity decoupling issue, where directly adopting existing customization methods inevitably mix identity attributes when handling multiple concepts simultaneously, and 2) the scarcity of high-quality video-entity pairs, which is crucial for training a model that can well represent and decouple various customized concepts in video generation. To address these challenges, we introduce ConceptMaster, a novel framework that effectively addresses the identity decoupling issues while maintaining concept fidelity in video customization. Specifically, we propose to learn decoupled multi-concept embeddings and inject them into diffusion models in a standalone manner, which effectively guarantees the quality of customized videos with multiple identities, even for highly similar visual concepts. To overcome the scarcity of high-quality MCVC data, we establish a data construction pipeline, which enables collection of high-quality multi-concept video-entity data pairs across diverse scenarios. A multi-concept video evaluation set is further devised to comprehensively validate our method from three dimensions, including concept fidelity, identity decoupling ability, and video generation quality, across six different concept composition scenarios. Extensive experiments demonstrate that ConceptMaster significantly outperforms previous methods for video customization tasks, showing great potential to generate personalized and semantically accurate content for video diffusion models.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 64 of its 71 references that are in the map. First posted on arXiv 8 Jan 2025.

Builds on

64 of 71 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  2. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motion7 Dec
  3. 2023VideoBooth: Diffusion-based Video Generation with Image Prompts1 Dec
  4. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models7 Nov
  5. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors18 Oct
  6. 2023ModelScope Text-to-Video Technical Report12 Aug
  7. 2023AnyDoor: Zero-shot Object-level Image Customization18 Jul
  8. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  10. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  11. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation15 Mar
  12. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
2 more
  1. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  2. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Models20 Nov

Other references · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2023Segment Anythingcited by 8,158
  11. 2023Visual Instruction Tuningcited by 7,792
  12. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
38 more
  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  2. 2022Scalable Diffusion Models with Transformerscited by 5,931
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  11. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  13. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  14. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  15. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  16. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  17. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  18. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  19. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  20. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  21. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  22. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  23. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  24. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  25. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  26. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  27. 2023Scaling Open-Vocabulary Object Detectioncited by 335
  28. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
  29. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  30. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  31. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
  32. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  33. 2023Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuningcited by 191
  34. 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 142
  35. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  36. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 122
  37. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Modelscited by 54
  38. 2024Story3D-Agent: Exploring 3D Storytelling Visualization with Large Language Modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 42; 2026: 25.

67 papers.

Sort:

By area:

  1. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
  2. 2025Long Context Tuning for Video Generationcited by 110
  3. 2025Controllable Video Generation: A Surveycited by 93
  4. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  5. 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 84
  6. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  7. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  8. 2025HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narrativescited by 51
  9. 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 45
  10. 2025OneStory: Coherent Multi-Shot Video Generation with Adaptive Memorycited by 31
  11. 2025Concat-ID: Towards Universal Identity-Preserving Video Synthesiscited by 28
  12. 2025DreamRelation: Relation-Centric Video Customizationcited by 23
38 more
  1. 2025MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglementcited by 22
  2. 2025BindWeave: Subject-Consistent Video Generation via Cross-Modal Integrationcited by 22
  3. 2025InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio Conditionscited by 20
  4. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generationcited by 20
  5. 2025CINEMA: Coherent Multi-Subject Video Generation via MLLM-Based Guidancecited by 19
  6. 2025DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformerscited by 19
  7. 2026DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generationcited by 19
  8. 2025Phantom-Data : Towards a General Subject-Consistent Video Generation Datasetcited by 17
  9. 2025OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditionscited by 17
  10. 2025PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancementcited by 16
  11. 2025Cut2Next: Generating Next Shot via In-Context Tuningcited by 15
  12. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  13. 2025Video-As-Prompt: Unified Semantic Control for Video Generationcited by 13
  14. 2025Tora2: Motion and Appearance Customized Diffusion Transformer for Multi-Entity Video Generationcited by 12
  15. 2025Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Modelscited by 10
  16. 2026RefAlign: Representation Alignment for Reference-to-Video Generationcited by 10
  17. 2025OmniInsert: Mask-Free Video Insertion of Any Reference via Diffusion Transformer Modelscited by 9
  18. 2025Identity-GRPO: Optimizing Multi-Human Identity-preserving Video Generation via Reinforcement Learningcited by 9
  19. 2025FilmWeaver: Weaving Consistent Multi-Shot Videos with Cache-Guided Autoregressive Diffusioncited by 9
  20. 2026MV-S2V: Multi-View Subject-Consistent Video Generationcited by 9
  21. 2025Any2Caption:Interpreting Any Condition to Caption for Controllable Video Generationcited by 8
  22. 2025AnimeShooter: A Multi-Shot Animation Dataset for Reference-Guided Video Generationcited by 8
  23. 2025Lynx: Towards High-Fidelity Personalized Video Generationcited by 8
  24. 2025Bind-Your-Avatar: Multi-Talking-Character Video Generation with Dynamic 3D-mask-based Embedding Routercited by 6
  25. 2025A Survey on Long-Video Storytelling Generation: Architectures, Consistency, and Cinematic Qualitycited by 6
  26. 2025MoFu: Scale-Aware Modulation and Fourier Fusion for Multi-Subject Video Generationcited by 6
  27. 2026OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfercited by 6
  28. 2026LumosX: Relate Any Identities with Their Attributes for Personalized Video Generationcited by 6
  29. 2026Evolution of Video Generative Foundationscited by 6
  30. 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 6
  31. 2025From Generated Human Videos to Physically Plausible Robot Trajectoriescited by 5
  32. 2025ContextAnyone: Context-Aware Diffusion for Character-Consistent Text-to-Video Generationcited by 5
  33. 2026DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learningcited by 4
  34. 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generationcited by 4
  35. 2025AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generationcited by 3
  36. 2025Over++: Generative Video Compositing for Layer Interaction Effectscited by 3
  37. 2026Tuning-free Visual Effect Transfer across Videoscited by 3
  38. 2026OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Modelcited by 3

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities
  2. 2024TweedieMix: Improving Multi-Concept Fusion for Diffusion-based Image/Video Generation
  3. 2025Dynamic Concepts Personalization from Single Videos
  4. 2026A Comprehensive Ecosystem for Open-Domain Customized Video Generation
  5. 2025VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Models
  6. 2025Ingredients: Blending Custom Photos with Video Diffusion Transformers
  7. 2025MoCA: Identity-Preserving Text-to-Video Generation via Mixture of Cross Attention
  8. 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjects