mlsanityData to 14 Oct 2026

OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation

arXiv 2505.20292 ↗First posted 26 May 2025cs.CV cs.AIcited by 73

Not on the map. Why?

Abstract (arXiv)

Subject-to-Video (S2V) generation aims to create videos that faithfully incorporate reference content, providing enhanced flexibility in the production of videos. To establish the infrastructure for S2V generation, we propose OpenS2V-Nexus, consisting of (i) OpenS2V-Eval, a fine-grained benchmark, and (ii) OpenS2V-5M, a million-scale dataset. In contrast to existing S2V benchmarks inherited from VBench that focus on global and coarse-grained assessment of generated videos, OpenS2V-Eval focuses on the model's ability to generate subject-consistent videos with natural subject appearance and identity fidelity. For these purposes, OpenS2V-Eval introduces 180 prompts from seven major categories of S2V, which incorporate both real and synthetic test data. Furthermore, to accurately align human preferences with S2V benchmarks, we propose three automatic metrics, NexusScore, NaturalScore and GmeScore, to separately quantify subject consistency, naturalness, and text relevance in generated videos. Building on this, we conduct a comprehensive evaluation of 18 representative S2V models, highlighting their strengths and weaknesses across different content. Moreover, we create the first open-source large-scale S2V generation dataset OpenS2V-5M, which consists of five million high-quality 720P subject-text-video triples. Specifically, we ensure subject-information diversity in our dataset by (1) segmenting subjects and building pairing information via cross-video associations and (2) prompting GPT-Image-1 on raw frames to synthesize multi-view representations. Through OpenS2V-Nexus, we deliver a robust infrastructure to accelerate future S2V generation research.

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). Builds on 106 of its 126 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

106 of 126 references.

References · 106

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Classifier-Free Diffusion Guidancecited by 5,847
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  12. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
38 more
  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  4. 2025Gemma 3 Technical Reportcited by 1,824
  5. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  7. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  8. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  10. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  11. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  12. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  13. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
  14. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
  15. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  16. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  17. 2025VACE: All-in-One Video Creation and Editingcited by 479
  18. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  19. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
  20. 2024OmniGen: Unified Image Generationcited by 394
  21. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  22. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  23. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  24. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  25. 2020CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognitioncited by 321
  26. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  27. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  28. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  29. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  30. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 294
  31. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 250
  32. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 197
  33. 2024A Sanity Check for AI-generated Image Detectioncited by 189
  34. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 188
  35. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 174
  36. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
  37. 2025Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Modelcited by 168
  38. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 163

50 most cited of 106 All 106 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 52.

73 papers.

Sort:

By area:

  1. 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 66
  2. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  3. 2026Rethinking Video Generation Model for the Embodied Worldcited by 33
  4. 2025Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generationcited by 28
  5. 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoningcited by 27
  6. 2026VINO: A Unified Visual Generator with Interleaved OmniModal Contextcited by 26
  7. 2025MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglementcited by 23
  8. 2025BindWeave: Subject-Consistent Video Generation via Cross-Modal Integrationcited by 23
  9. 2025Region-Constraint In-Context Generation for Instructional Video Editingcited by 23
  10. 2025UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalistcited by 15
  11. 2025Reinforcement Learning with Inverse Rewards for World Model Post-trainingcited by 13
  12. 2026MSVBench: Towards Human-Level Evaluation of Multi-Shot Video Generationcited by 12
38 more
  1. 2026Bernini: Latent Semantic Planning for Video Diffusioncited by 12
  2. 2025Identity-GRPO: Optimizing Multi-Human Identity-preserving Video Generation via Reinforcement Learningcited by 10
  3. 2026RefAlign: Representation Alignment for Reference-to-Video Generationcited by 10
  4. 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 10
  5. 2025ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraintscited by 9
  6. 2025Scaling Zero-Shot Reference-to-Video Generationcited by 9
  7. 2026MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generationcited by 9
  8. 2026Aurora: Unified Video Editing with a Tool-Using Agentcited by 9
  9. 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editingcited by 8
  10. 2026OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generationcited by 8
  11. 2025Video Generation with Stable Transparency via Shiftable RGB-A Distribution Learnercited by 7
  12. 2025UnityVideo: Unified Multi-Modal Multi-Task Learning for Enhancing World-Aware Video Generationcited by 7
  13. 2026ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generationcited by 7
  14. 2025ID-Crafter: VLM-Grounded Online RL for Compositional Multi-Subject Video Generationcited by 6
  15. 2025MoFu: Scale-Aware Modulation and Fourier Fusion for Multi-Subject Video Generationcited by 6
  16. 2026EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generationcited by 6
  17. 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 6
  18. 2026MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generationcited by 6
  19. 2026FashionChameleon: Towards Real-Time and Interactive Human-Garment Video Customizationcited by 5
  20. 2026VideoArgus: Agentic Rubric-Grounded Unified Evaluation for Video Generation and Editingcited by 5
  21. 2025FlashI2V: Fourier-Guided Latent Shifting Prevents Conditional Image Leakage in Image-to-Video Generationcited by 4
  22. 2025OpenSubject: Leveraging Video-Derived Identity and Diversity Priors for Subject-driven Image Generation and Manipulationcited by 4
  23. 2025TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detectioncited by 4
  24. 2026Slot-ID: Identity-Preserving Video Generation from Reference Videos via Slot-Based Temporal Identity Encodingcited by 4
  25. 2026WildActor: Unconstrained Identity-Preserving Video Generationcited by 4
  26. 2025AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generationcited by 3
  27. 2026Identity-Consistent Video Generation under Large Facial-Angle Variationscited by 3
  28. 2026From Understanding to Erasing: Towards Complete and Stable Video Object Removalcited by 3
  29. 2026Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generationcited by 3
  30. 2025What Happens Next? Next Scene Prediction with a Unified Video Modelcited by 2
  31. 2026ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformercited by 2
  32. 2026Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generationcited by 2
  33. 2026Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Modelscited by 2
  34. 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generationcited by 2
  35. 2026Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignmentcited by 2
  36. 2025DreamingComics: A Story Visualization Pipeline via Subject and Layout Customized Generation using Video Modelscited by 1
  37. 2025Dynamic Image Prompt Adapter for Scalable Zero-shot Personalized Text-to-Image Generationcited by 1
  38. 2026PLACID: Identity-Preserving Multi-Object Compositing via Video Diffusion with Synthetic Trajectoriescited by 1

50 most cited of 73 All 73 (JSON)