# PolyVivid: Vivid Multi-Subject Video Generation with Cross-Modal Interaction and Enhancement

arXiv: https://arxiv.org/abs/2506.07848 · HTML: https://mlsanity.com/paper/2506.07848/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 54 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Builds on (most cited first, 30 of 43)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023)
- [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers](https://mlsanity.com/paper/2402.19479/) (2024)
- [OminiControl: Minimal and Universal Control for Diffusion Transformer](https://mlsanity.com/paper/2411.15098/) (2024)
- [Identity-Preserving Text-to-Video Generation by Frequency Decomposition](https://mlsanity.com/paper/2411.17440/) (2024)
- [Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content](https://mlsanity.com/paper/2410.08260/) (2024)
- [Phantom: Subject-consistent video generation via cross-modal alignment](https://mlsanity.com/paper/2502.11079/) (2025)
- [VideoBooth: Diffusion-based Video Generation with Image Prompts](https://mlsanity.com/paper/2312.00777/) (2023)

## Built on by (most cited first, 16 of 16)

- [UltraGen: High-Resolution Video Generation with Hierarchical Attention](https://mlsanity.com/paper/2510.18775/) (2025)
- [RefAlign: Representation Alignment for Reference-to-Video Generation](https://mlsanity.com/paper/2603.25743/) (2026)
- [MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data](https://mlsanity.com/paper/2606.02753/) (2026)
- [Scaling Zero-Shot Reference-to-Video Generation](https://mlsanity.com/paper/2512.06905/) (2025)
- [Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding](https://mlsanity.com/paper/2604.11244/) (2026)
- [Evolution of Video Generative Foundations](https://mlsanity.com/paper/2604.06339/) (2026)
- [ID-Crafter: VLM-Grounded Online RL for Compositional Multi-Subject Video Generation](https://mlsanity.com/paper/2511.00511/) (2025)
- [PoseAnything: Universal Pose-guided Video Generation with Part-aware Temporal Coherence](https://mlsanity.com/paper/2512.13465/) (2025)
- [PresentAgent-2: Towards Generalist Multimodal Presentation Agents](https://mlsanity.com/paper/2605.11363/) (2026)
- [InstanceV: Instance-Level Video Generation](https://mlsanity.com/paper/2511.23146/) (2025)
- [Gloria: Consistent Character Video Generation via Content Anchors](https://mlsanity.com/paper/2603.29931/) (2026)
- [Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment](https://mlsanity.com/paper/2607.04311/) (2026)
- [Cycle-World: Mitigating Error Accumulation in Long-term Video World Models via Reverse-Prediction Cycle Consistency](https://mlsanity.com/paper/2607.11836/) (2026)
- [HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement](https://mlsanity.com/paper/2607.18217/) (2026)
- [Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation](https://mlsanity.com/paper/2606.02441/) (2026)
- [Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation](https://mlsanity.com/paper/2609.11507/) (2026)

## Abstract (arXiv)

Despite recent advances in video generation, existing models still lack fine-grained controllability, especially for multi-subject customization with consistent identity and interaction. In this paper, we propose PolyVivid, a multi-subject video customization framework that enables flexible and identity-consistent generation. To establish accurate correspondences between subject images and textual entities, we design a VLLM-based text-image fusion module that embeds visual identities into the textual space for precise grounding. To further enhance identity preservation and subject interaction, we propose a 3D-RoPE-based enhancement module that enables structured bidirectional fusion between text and image embeddings. Moreover, we develop an attention-inherited identity injection module to effectively inject fused identity features into the video generation process, mitigating identity drift. Finally, we construct an MLLM-based data pipeline that combines MLLM-based grounding, segmentation, and a clique-based subject consolidation strategy to produce high-quality multi-subject data, effectively enhancing subject distinction and reducing ambiguity in downstream video generation. Extensive experiments demonstrate that PolyVivid achieves superior performance in identity fidelity, video realism, and subject alignment, outperforming existing open-source and commercial baselines.