mlsanityData to 14 Oct 2026

World Consistency Score: A Unified Metric for Video Generation Quality

arXiv 2508.00144 ↗First posted 31 Jul 2025cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

We introduce World Consistency Score (WCS), a novel unified evaluation metric for generative video models that emphasizes internal world consistency of the generated videos. WCS integrates four interpretable sub-components - object permanence, relation stability, causal compliance, and flicker penalty - each measuring a distinct aspect of temporal and physical coherence in a video. These submetrics are combined via a learned weighted formula to produce a single consistency score that aligns with human judgments. We detail the motivation for WCS in the context of existing video evaluation metrics, formalize each submetric and how it is computed with open-source tools (trackers, action recognizers, CLIP embeddings, optical flow), and describe how the weights of the WCS combination are trained using human preference data. We also outline an experimental validation blueprint: using benchmarks like VBench-2.0, EvalCrafter, and LOVE to test WCS's correlation with human evaluations, performing sensitivity analyses, and comparing WCS against established metrics (FVD, CLIPScore, VBench, FVMD). The proposed WCS offers a comprehensive and interpretable framework for evaluating video generation models on their ability to maintain a coherent "world" over time, addressing gaps left by prior metrics focused only on visual fidelity or prompt alignment.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Jul 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Video-Bench: Human-Aligned Video Generation Benchmark
  2. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness
  3. 2025WorldModelBench: Judging Video Generation Models As World Models
  4. 2025VideoScore2: Think before You Score in Generative Video Evaluation
  5. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models
  6. 2024Towards A Better Metric for Text-to-Video Generation
  7. 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation
  8. 2026VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality