# OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation

arXiv: https://arxiv.org/abs/2609.22069 · HTML: https://mlsanity.com/paper/2609.22069/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 40 of its 73 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Builds on (most cited first, 30 of 40)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation](https://mlsanity.com/paper/2407.02371/) (2024)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [Unified Reward Model for Multimodal Understanding and Generation](https://mlsanity.com/paper/2503.05236/) (2025)
- [TransNet V2: An effective deep network architecture for fast shot transition detection](https://mlsanity.com/paper/2008.04838/) (2020)
- [Pixel Difference Networks for Efficient Edge Detection](https://mlsanity.com/paper/2108.07009/) (2021)
- [Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Models](https://mlsanity.com/paper/2405.04233/) (2024)
- [HunyuanVideo 1.5 Technical Report](https://mlsanity.com/paper/2511.18870/) (2025)
- [MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions](https://mlsanity.com/paper/2407.06358/) (2024)
- [Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content](https://mlsanity.com/paper/2410.08260/) (2024)
- [HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation](https://mlsanity.com/paper/2505.04512/) (2025)
- [UniVideo: Unified Understanding, Generation, and Editing for Videos](https://mlsanity.com/paper/2510.08377/) (2025)
- [Learning to generate line drawings that convey geometry and semantics](https://mlsanity.com/paper/2203.12691/) (2022)
- [Wan-Animate: Unified Character Animation and Replacement with Holistic Replication](https://mlsanity.com/paper/2509.14055/) (2025)
- [OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation](https://mlsanity.com/paper/2412.00115/) (2024)
- [OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation](https://mlsanity.com/paper/2505.20292/) (2025)
- [Multi-subject Open-set Personalization in Video Generation](https://mlsanity.com/paper/2501.06187/) (2025)
- [Waver: Wave Your Way to Lifelike Video Generation](https://mlsanity.com/paper/2508.15761/) (2025)
- [Tiny and Efficient Model for the Edge Detection Generalization](https://mlsanity.com/paper/2308.06468/) (2023)
- [Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generation](https://mlsanity.com/paper/2508.07901/) (2025)
- [VINO: A Unified Visual Generator with Interleaved OmniModal Context](https://mlsanity.com/paper/2601.02358/) (2026)
- [DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation](https://mlsanity.com/paper/2602.12160/) (2026)
- [Phantom-Data : Towards a General Subject-Consistent Video Generation Dataset](https://mlsanity.com/paper/2506.18851/) (2025)
- [OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions](https://mlsanity.com/paper/2506.23361/) (2025)

## Abstract (arXiv)

Omni reference-to-video (R2V) generation is emerging as a general paradigm that enables flexible video customization from diverse and compositional visual references. However, existing benchmarks fall short of these emerging capabilities: their test cases cover only a limited range of reference types and compositions, and their evaluation protocols largely assess holistic reference consistency, overlooking whether reference factors are properly preserved, disentangled, and routed. Meanwhile, the high cost of constructing omni R2V training data makes suitable training resources scarce. To address these gaps, we introduce OmniVBench and the Omni-R2V Dataset, establishing a shared foundation for evaluating and training omni R2V models. OmniVBench expands R2V evaluation across broader reference types, richer reference compositions, and more flexible instruction-driven customization, covering 7 task families and 18 fine-grained tasks. Complementing this task design, we introduce factor-grounded evaluation with 12,172 case-specific checklist items, explicitly assessing whether intended reference factors are faithfully preserved, correctly disentangled and bound to their targets, and realized according to the instruction. We further introduce the Omni-R2V Dataset, bringing industrial-grade training resources for diverse R2V tasks to the broader research community. Drawing primarily on a large-scale corpus of professional video footage, it comprises 340K processed training samples spanning diverse reference types and multi-reference compositions. Specifically, we develop task-specific pipelines for reference-target pair construction, offering a practical and scalable recipe for omni R2V data construction. Extensive evaluation of advanced open- and closed-source R2V models reveals substantial performance variation across diverse tasks and distinct capability patterns across evaluation dimensions.