# Kaleido: Open-Sourced Multi-Subject Reference Video Generation Model

arXiv: https://arxiv.org/abs/2510.18573 · HTML: https://mlsanity.com/paper/2510.18573/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 21 references that are in the map.
- First posted on arXiv 21 Oct 2025.

## Builds on (most cited first, 19 of 19)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation](https://mlsanity.com/paper/2307.06942/) (2023)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models](https://mlsanity.com/paper/2405.04233/) (2024)
- [Phantom: Subject-consistent video generation via cross-modal alignment](https://mlsanity.com/paper/2502.11079/) (2025)
- [VideoBooth: Diffusion-based Video Generation with Image Prompts](https://mlsanity.com/paper/2312.00777/) (2023)
- [CustomVideo: Customizing Text-to-Video Generation with Multiple Subjects](https://mlsanity.com/paper/2401.09962/) (2024)
- [SkyReels-A2: Compose Anything in Video Diffusion Transformers](https://mlsanity.com/paper/2504.02436/) (2025)
- [SUGAR: Subject-Driven Video Customization in a Zero-Shot Manner](https://mlsanity.com/paper/2412.10533/) (2024)

## Built on by (most cited first, 16 of 16)

- [RefAlign: Representation Alignment for Reference-to-Video Generation](https://mlsanity.com/paper/2603.25743/) (2026)
- [MV-S2V: Multi-View Subject-Consistent Video Generation](https://mlsanity.com/paper/2601.17756/) (2026)
- [MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generation](https://mlsanity.com/paper/2604.23789/) (2026)
- [FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customization](https://mlsanity.com/paper/2605.15824/) (2026)
- [VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editing](https://mlsanity.com/paper/2608.05485/) (2026)
- [CineWeaver: Training-Free Reference-Controllable Multi-Shot Long Video Generation for Cinematic Storytelling](https://mlsanity.com/paper/2607.26529/) (2026)
- [MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model](https://mlsanity.com/paper/2603.14686/) (2026)
- [Identity-Consistent Video Generation under Large Facial-Angle Variations](https://mlsanity.com/paper/2603.21299/) (2026)
- [LibraGen: Playing a Balance Game in Subject-Driven Video Generation](https://mlsanity.com/paper/2603.13506/) (2026)
- [Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generation](https://mlsanity.com/paper/2605.17488/) (2026)
- [Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment](https://mlsanity.com/paper/2607.04311/) (2026)
- [HarmoView: Harmonizing Multi-View Constraints for Identity-Consistent Video Generation](https://mlsanity.com/paper/2606.10839/) (2026)
- [DISPLAY: Directable Human-Object Interaction Video Generation via Sparse Motion Guidance and Multi-Task Auxiliary](https://mlsanity.com/paper/2603.09883/) (2026)
- [Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation](https://mlsanity.com/paper/2606.02441/) (2026)
- [Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation](https://mlsanity.com/paper/2609.11507/) (2026)
- [AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generation](https://mlsanity.com/paper/2610.10047/) (2026)

## Abstract (arXiv)

We present Kaleido, a subject-to-video~(S2V) generation framework, which aims to synthesize subject-consistent videos conditioned on multiple reference images of target subjects. Despite recent progress in S2V generation models, existing approaches remain inadequate at maintaining multi-subject consistency and at handling background disentanglement, often resulting in lower reference fidelity and semantic drift under multi-image conditioning. These shortcomings can be attributed to several factors. Primarily, the training dataset suffers from a lack of diversity and high-quality samples, as well as cross-paired data, i.e., paired samples whose components originate from different instances. In addition, the current mechanism for integrating multiple reference images is suboptimal, potentially resulting in the confusion of multiple subjects. To overcome these limitations, we propose a dedicated data construction pipeline, incorporating low-quality sample filtering and diverse data synthesis, to produce consistency-preserving training data. Moreover, we introduce Reference Rotary Positional Encoding (R-RoPE) to process reference images, enabling stable and precise multi-image integration. Extensive experiments across numerous benchmarks demonstrate that Kaleido significantly outperforms previous methods in consistency, fidelity, and generalization, marking an advance in S2V generation.