HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation
Not on the map. Why?
Abstract (arXiv)
Customized video generation aims to produce videos featuring specific subjects under flexible user-defined conditions, yet existing methods often struggle with identity consistency and limited input modalities. In this paper, we propose HunyuanCustom, a multi-modal customized video generation framework that emphasizes subject consistency while supporting image, audio, video, and text conditions. Built upon HunyuanVideo, our model first addresses the image-text conditioned generation task by introducing a text-image fusion module based on LLaVA for enhanced multi-modal understanding, along with an image ID enhancement module that leverages temporal concatenation to reinforce identity features across frames. To enable audio- and video-conditioned generation, we further propose modality-specific condition injection mechanisms: an AudioNet module that achieves hierarchical alignment via spatial cross-attention, and a video-driven injection module that integrates latent-compressed conditional video through a patchify-based feature-alignment network. Extensive experiments on single- and multi-subject scenarios demonstrate that HunyuanCustom significantly outperforms state-of-the-art open- and closed-source methods in terms of ID consistency, realism, and text-video alignment. Moreover, we validate its robustness across downstream tasks, including audio and video-driven customized video generation. Our results highlight the effectiveness of multi-modal conditioning and identity-preserving strategies in advancing controllable video generation. All the code and models are available at https://hunyuancustom.github.io.
Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 52 of its 57 references that are in the map. First posted on arXiv 7 May 2025.
Builds on
52 of 57 references.
References · 52
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2023Qwen Technical Reportcited by 3,273
38 more
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
- 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
- 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
- 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
- 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 132
- 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
- 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 124
- 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 122
- 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
- 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
- 2024T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergycited by 94
- 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
- 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjectscited by 76
- 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
- 2024Sonic: Shifting Focus to Global Audio Perception in Portrait Animationcited by 70
- 2024Allegro: Open the Black Box of Commercial-Level Video Generation Modelcited by 68
- 2025ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuningcited by 67
- 2025Multi-subject Open-set Personalization in Video Generationcited by 65
- 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
- 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 58
- 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animationcited by 55
- 2024LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervisioncited by 53
- 2024MotionMaster: Training-free Camera Motion Transfer For Video Generationcited by 50
- 2024Still-Moving: Customized Video Generation without Customized Video Datacited by 49
- 2023High-Fidelity and Freely Controllable Talking Head Video Generationcited by 42
- 2022Arbitrary Shape Text Detection via Boundary Transformercited by 28
- 2022Facial Geometric Detail Recovery via Implicit Representationcited by 27
- 2024Make-Your-Anchor: A Diffusion-based 2D Avatar Generation Frameworkcited by 26
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2025: 35; 2026: 49.
84 papers.
Sort:
By area:
- 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
- 2025UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactionscited by 35
- 2025LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuningcited by 27
- 2025Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generationcited by 27
- 2025MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglementcited by 22
- 2025BindWeave: Subject-Consistent Video Generation via Cross-Modal Integrationcited by 22
- 2025Region-Constraint In-Context Generation for Instructional Video Editingcited by 22
- 2025DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformerscited by 19
- 2026DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generationcited by 19
- 2025OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulationcited by 18
- 2025Phantom-Data : Towards a General Subject-Consistent Video Generation Datasetcited by 17
- 2025OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditionscited by 17
38 more
- 2025UltraGen: High-Resolution Video Generation with Hierarchical Attentioncited by 17
- 2025PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancementcited by 16
- 2025DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Frameworkcited by 12
- 2026ReRoPE: Repurposing RoPE for Relative Camera Controlcited by 12
- 2025ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modelingcited by 11
- 2026RefAlign: Representation Alignment for Reference-to-Video Generationcited by 10
- 2025GenCompositor: Generative Video Compositing with Diffusion Transformercited by 9
- 2025Identity-GRPO: Optimizing Multi-Human Identity-preserving Video Generation via Reinforcement Learningcited by 9
- 2025Scaling Zero-Shot Reference-to-Video Generationcited by 9
- 2025PoseGen: In-Context LoRA Finetuning for Pose-Controllable Long Human Video Generationcited by 8
- 2025Lynx: Towards High-Fidelity Personalized Video Generationcited by 8
- 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillationcited by 8
- 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 8
- 2026OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generationcited by 7
- 2026ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generationcited by 7
- 2025A Survey on Long-Video Storytelling Generation: Architectures, Consistency, and Cinematic Qualitycited by 6
- 2026MoCha:End-to-End Video Character Replacement without Structural Guidancecited by 6
- 2026OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfercited by 6
- 2026Evolution of Video Generative Foundationscited by 6
- 2026Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generationcited by 6
- 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 6
- 2025From Generated Human Videos to Physically Plausible Robot Trajectoriescited by 5
- 2025ContextAnyone: Context-Aware Diffusion for Character-Consistent Text-to-Video Generationcited by 5
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 5
- 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editingcited by 5
- 2025Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representationscited by 4
- 2025MultiCrafter: High-Fidelity Multi-Subject Generation via Disentangled Attention and Identity-Aware Preference Alignmentcited by 4
- 2025TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detectioncited by 4
- 2025PoseAnything: Universal Pose-guided Video Generation with Part-aware Temporal Coherencecited by 4
- 2026Slot-ID: Identity-Preserving Video Generation from Reference Videos via Slot-Based Temporal Identity Encodingcited by 4
- 2026TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generationcited by 4
- 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generationcited by 4
- 2025Soul: Breathe Life into Digital Human for High-fidelity Long-term Multimodal Animationcited by 3
- 2026Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatarscited by 3
- 2026OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Modelcited by 3
- 2026TRACE: Object Motion Editing in Videos with First-Frame Trajectory Guidancecited by 3
- 2026Gloria: Consistent Character Video Generation via Content Anchorscited by 3
- 2026Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generationcited by 3
50 most cited of 84 All 84 (JSON)
- 2026AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generation7 Oct 2026
- 2026OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation18 Sep 2026
- 2026Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation10 Sep 2026
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention16 Aug 2026
- 2026UniSwap: Streaming Audio-Visual Identity Swapping for Talking Videos12 Aug 2026
- 2026Sci-VBench: Evaluating Knowledge- and Reasoning-Intensive Video Generation in Science Domains10 Aug 2026
- 2026Vorch-IR: Long-Form Unified Multimodal Identity Replacement Video Generation6 Aug 2026
- 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing6 Aug 2026
- 2026UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation3 Aug 2026
- 2026CoT-Edit: Let CoT Guide Instruction Video Editing2 Aug 2026
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement20 Jul 2026
- 2026Keyframe-Anchored Identity Preservation for Sequential-Action Video Generation20 Jul 2026
38 more
- 2026Cycle-World: Mitigating Error Accumulation in Long-term Video World Models via Reverse-Prediction Cycle Consistency13 Jul 2026
- 2026Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment5 Jul 2026
- 2026Ink3D: Sculpting 3D Assets with Extremely Complex Textures via Video Generative Models1 Jul 2026
- 2026DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation24 Jun 2026
- 2026Go-with-the-Track: Video Compositing and Motion Control with Point Tracking18 Jun 2026
- 2026ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation10 Jun 2026
- 2026HarmoView: Harmonizing Multi-View Constraints for Identity-Consistent Video Generation9 Jun 2026
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation8 Jun 2026
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation6 Jun 2026
- 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data1 Jun 2026
- 2026Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation1 Jun 2026
- 2026Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generation17 May 2026
- 2026UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation12 May 2026
- 2026FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation6 May 2026
- 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation21 Apr 2026
- 2026TS-Attn: Temporal-wise Separable Attention for Multi-Event Video Generation21 Apr 2026
- 2026Controllable Video Object Insertion via Multi-View Priors16 Apr 2026
- 2026OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation13 Apr 2026
- 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding13 Apr 2026
- 2026Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generation11 Apr 2026
- 2026Evolution of Video Generative Foundations7 Apr 2026
- 2026Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation4 Apr 2026
- 2026Gloria: Consistent Character Video Generation via Content Anchors31 Mar 2026
- 2026TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Space29 Mar 2026
- 2026RefAlign: Representation Alignment for Reference-to-Video Generation26 Mar 2026
- 2026TRACE: Object Motion Editing in Videos with First-Frame Trajectory Guidance26 Mar 2026
- 2026Identity as Presence: Towards Appearance and Voice Personalized Joint Audio-Video Generation18 Mar 2026
- 2026MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model16 Mar 2026
- 2026DISPLAY: Directable Human-Object Interaction Video Generation via Sparse Motion Guidance and Multi-Task Auxiliary10 Mar 2026
- 2026FlexAM: Flexible Appearance-Motion Decomposition for Versatile Video Generation Control13 Feb 2026
- 2026OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Model12 Feb 2026
- 2026DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation12 Feb 2026
- 2026ReRoPE: Repurposing RoPE for Relative Camera Control8 Feb 2026
- 2026Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatars2 Feb 2026
- 2026OmniTransfer: All-in-one Framework for Spatio-temporal Video Transfer20 Jan 2026
- 2026MoCha:End-to-End Video Character Replacement without Structural Guidance13 Jan 2026
- 2026Slot-ID: Identity-Preserving Video Generation from Reference Videos via Slot-Based Temporal Identity Encoding4 Jan 2026
- 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillation29 Dec 2025
50 newest of 84 All 84 (JSON)
- 2025LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuningcited by 27
- 2025BindWeave: Subject-Consistent Video Generation via Cross-Modal Integrationcited by 22
- 2025DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformerscited by 19
- 2025OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulationcited by 18
- 2025DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Frameworkcited by 12
- 2026ReRoPE: Repurposing RoPE for Relative Camera Controlcited by 12
- 2025ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modelingcited by 11
- 2025GenCompositor: Generative Video Compositing with Diffusion Transformercited by 9
- 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillationcited by 8
- 2026Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generationcited by 6
- 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generationcited by 4
- 2025InstanceV: Instance-Level Video Generationcited by 2
12 most cited of 15 All 15 (JSON)
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation6 Jun 2026
- 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation21 Apr 2026
- 2026Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generation11 Apr 2026
- 2026TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Space29 Mar 2026
- 2026ReRoPE: Repurposing RoPE for Relative Camera Control8 Feb 2026
- 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillation29 Dec 2025
- 2025V-Warper: Appearance-Consistent Video Diffusion Personalization via Value Warping13 Dec 2025
- 2025InstanceV: Instance-Level Video Generation28 Nov 2025
- 2025ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modeling10 Oct 2025
- 2025BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration1 Oct 2025
- 2025GenCompositor: Generative Video Compositing with Diffusion Transformer2 Sep 2025
- 2025DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Framework4 Aug 2025
12 newest of 15 All 15 (JSON)
- 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 8
- 2026Evolution of Video Generative Foundationscited by 6
- 2025CustomX: Unified Character, Action, and Scene Customization in Video World Modelscited by 1
- 2026Cycle-World: Mitigating Error Accumulation in Long-term Video World Models via Reverse-Prediction Cycle Consistencycited by 1
- 2026Cycle-World: Mitigating Error Accumulation in Long-term Video World Models via Reverse-Prediction Cycle Consistency13 Jul 2026
- 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data1 Jun 2026
- 2026Evolution of Video Generative Foundations7 Apr 2026
- 2025CustomX: Unified Character, Action, and Scene Customization in Video World Models18 Dec 2025
- 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
- 2025UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactionscited by 35
- 2025Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generationcited by 27
- 2025MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglementcited by 22
- 2025Region-Constraint In-Context Generation for Instructional Video Editingcited by 22
- 2026DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generationcited by 19
- 2025Phantom-Data : Towards a General Subject-Consistent Video Generation Datasetcited by 17
- 2025OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditionscited by 17
- 2025UltraGen: High-Resolution Video Generation with Hierarchical Attentioncited by 17
- 2025PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancementcited by 16
- 2026RefAlign: Representation Alignment for Reference-to-Video Generationcited by 10
- 2025Identity-GRPO: Optimizing Multi-Human Identity-preserving Video Generation via Reinforcement Learningcited by 9
12 most cited of 61 All 61 (JSON)
- 2026AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generation7 Oct 2026
- 2026OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation18 Sep 2026
- 2026Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation10 Sep 2026
- 2026Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention16 Aug 2026
- 2026UniSwap: Streaming Audio-Visual Identity Swapping for Talking Videos12 Aug 2026
- 2026Vorch-IR: Long-Form Unified Multimodal Identity Replacement Video Generation6 Aug 2026
- 2026UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation3 Aug 2026
- 2026CoT-Edit: Let CoT Guide Instruction Video Editing2 Aug 2026
- 2026HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement20 Jul 2026
- 2026Keyframe-Anchored Identity Preservation for Sequential-Action Video Generation20 Jul 2026
- 2026Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment5 Jul 2026
- 2026Ink3D: Sculpting 3D Assets with Extremely Complex Textures via Video Generative Models1 Jul 2026
12 newest of 61 All 61 (JSON)