mlsanityData to 7 Oct 2026

HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation

arXiv 2505.04512 ↗First posted 7 May 2025cs.CVcited by 84

Not on the map. Why?

Abstract (arXiv)

Customized video generation aims to produce videos featuring specific subjects under flexible user-defined conditions, yet existing methods often struggle with identity consistency and limited input modalities. In this paper, we propose HunyuanCustom, a multi-modal customized video generation framework that emphasizes subject consistency while supporting image, audio, video, and text conditions. Built upon HunyuanVideo, our model first addresses the image-text conditioned generation task by introducing a text-image fusion module based on LLaVA for enhanced multi-modal understanding, along with an image ID enhancement module that leverages temporal concatenation to reinforce identity features across frames. To enable audio- and video-conditioned generation, we further propose modality-specific condition injection mechanisms: an AudioNet module that achieves hierarchical alignment via spatial cross-attention, and a video-driven injection module that integrates latent-compressed conditional video through a patchify-based feature-alignment network. Extensive experiments on single- and multi-subject scenarios demonstrate that HunyuanCustom significantly outperforms state-of-the-art open- and closed-source methods in terms of ID consistency, realism, and text-video alignment. Moreover, we validate its robustness across downstream tasks, including audio and video-driven customized video generation. Our results highlight the effectiveness of multi-modal conditioning and identity-preserving strategies in advancing controllable video generation. All the code and models are available at https://hunyuancustom.github.io.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 52 of its 57 references that are in the map. First posted on arXiv 7 May 2025.

Builds on

52 of 57 references.

References · 52

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  9. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  12. 2023Qwen Technical Reportcited by 3,273
38 more
  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  4. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  5. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  6. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  7. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  8. 2025VACE: All-in-One Video Creation and Editingcited by 467
  9. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
  10. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  11. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  12. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  13. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  14. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  15. 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 132
  16. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
  17. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 124
  18. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 122
  19. 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
  20. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
  21. 2024T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergycited by 94
  22. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  23. 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjectscited by 76
  24. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  25. 2024Sonic: Shifting Focus to Global Audio Perception in Portrait Animationcited by 70
  26. 2024Allegro: Open the Black Box of Commercial-Level Video Generation Modelcited by 68
  27. 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 67
  28. 2025Multi-subject Open-set Personalization in Video Generationcited by 65
  29. 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
  30. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 58
  31. 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animationcited by 55
  32. 2024LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervisioncited by 53
  33. 2024MotionMaster: Training-free Camera Motion Transfer For Video Generationcited by 50
  34. 2024Still-Moving: Customized Video Generation without Customized Video Datacited by 49
  35. 2023High-Fidelity and Freely Controllable Talking Head Video Generationcited by 42
  36. 2022Arbitrary Shape Text Detection via Boundary Transformercited by 28
  37. 2022Facial Geometric Detail Recovery via Implicit Representationcited by 27
  38. 2024Make-Your-Anchor: A Diffusion-based 2D Avatar Generation Frameworkcited by 26

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 35; 2026: 49.

84 papers.

Sort:

By area:

  1. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  2. 2025UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactionscited by 35
  3. 2025LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuningcited by 27
  4. 2025Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generationcited by 27
  5. 2025MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglementcited by 22
  6. 2025BindWeave: Subject-Consistent Video Generation via Cross-Modal Integrationcited by 22
  7. 2025Region-Constraint In-Context Generation for Instructional Video Editingcited by 22
  8. 2025DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformerscited by 19
  9. 2026DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generationcited by 19
  10. 2025OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulationcited by 18
  11. 2025Phantom-Data : Towards a General Subject-Consistent Video Generation Datasetcited by 17
  12. 2025OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditionscited by 17
38 more
  1. 2025UltraGen: High-Resolution Video Generation with Hierarchical Attentioncited by 17
  2. 2025PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancementcited by 16
  3. 2025DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Frameworkcited by 12
  4. 2026ReRoPE: Repurposing RoPE for Relative Camera Controlcited by 12
  5. 2025ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modelingcited by 11
  6. 2026RefAlign: Representation Alignment for Reference-to-Video Generationcited by 10
  7. 2025GenCompositor: Generative Video Compositing with Diffusion Transformercited by 9
  8. 2025Identity-GRPO: Optimizing Multi-Human Identity-preserving Video Generation via Reinforcement Learningcited by 9
  9. 2025Scaling Zero-Shot Reference-to-Video Generationcited by 9
  10. 2025PoseGen: In-Context LoRA Finetuning for Pose-Controllable Long Human Video Generationcited by 8
  11. 2025Lynx: Towards High-Fidelity Personalized Video Generationcited by 8
  12. 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillationcited by 8
  13. 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 8
  14. 2026OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generationcited by 7
  15. 2026ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generationcited by 7
  16. 2025A Survey on Long-Video Storytelling Generation: Architectures, Consistency, and Cinematic Qualitycited by 6
  17. 2026MoCha:End-to-End Video Character Replacement without Structural Guidancecited by 6
  18. 2026OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfercited by 6
  19. 2026Evolution of Video Generative Foundationscited by 6
  20. 2026Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generationcited by 6
  21. 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 6
  22. 2025From Generated Human Videos to Physically Plausible Robot Trajectoriescited by 5
  23. 2025ContextAnyone: Context-Aware Diffusion for Character-Consistent Text-to-Video Generationcited by 5
  24. 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 5
  25. 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editingcited by 5
  26. 2025Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representationscited by 4
  27. 2025MultiCrafter: High-Fidelity Multi-Subject Generation via Disentangled Attention and Identity-Aware Preference Alignmentcited by 4
  28. 2025TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detectioncited by 4
  29. 2025PoseAnything: Universal Pose-guided Video Generation with Part-aware Temporal Coherencecited by 4
  30. 2026Slot-ID: Identity-Preserving Video Generation from Reference Videos via Slot-Based Temporal Identity Encodingcited by 4
  31. 2026TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generationcited by 4
  32. 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generationcited by 4
  33. 2025Soul: Breathe Life into Digital Human for High-fidelity Long-term Multimodal Animationcited by 3
  34. 2026Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatarscited by 3
  35. 2026OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Modelcited by 3
  36. 2026TRACE: Object Motion Editing in Videos with First-Frame Trajectory Guidancecited by 3
  37. 2026Gloria: Consistent Character Video Generation via Content Anchorscited by 3
  38. 2026Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generationcited by 3

50 most cited of 84 All 84 (JSON)