# OSCBench: Benchmarking Object State Change in Text-to-Video Generation

arXiv: https://arxiv.org/abs/2603.11698 · HTML: https://mlsanity.com/paper/2603.11698/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Builds on (most cited first, 16 of 16)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [EvalCrafter: Benchmarking and Evaluating Large Video Generation Models](https://mlsanity.com/paper/2310.11440/) (2023)
- [VideoPhy: Evaluating Physical Commonsense for Video Generation](https://mlsanity.com/paper/2406.03520/) (2024)
- [VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation](https://mlsanity.com/paper/2406.15252/) (2024)
- [T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation](https://mlsanity.com/paper/2407.14505/) (2024)
- [Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation](https://mlsanity.com/paper/2410.05363/) (2024)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [Evaluation of Text-to-Video Generation Models: A Dynamics Perspective](https://mlsanity.com/paper/2407.01094/) (2024)
- [GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos](https://mlsanity.com/paper/2312.07322/) (2023)
- [Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos](https://mlsanity.com/paper/2203.11637/) (2022)
- [Learning Object State Changes in Videos: An Open-World Perspective](https://mlsanity.com/paper/2312.11782/) (2023)
- [MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration](https://mlsanity.com/paper/2505.23224/) (2025)
- [PunchBench: Benchmarking MLLMs in Multimodal Punchline Comprehension](https://mlsanity.com/paper/2412.11906/) (2024)
- [OSCAR: Object Status and Contextual Awareness for Recipes to Support Non-Visual Cooking](https://mlsanity.com/paper/2503.05962/) (2025)

## Built on by (most cited first, 3 of 3)

- [RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation](https://mlsanity.com/paper/2606.01600/) (2026)
- [YoCausal: How Far is Video Generation from World Model? A Causality Perspective](https://mlsanity.com/paper/2605.30346/) (2026)
- [Diagnosing Under-Development of Irreversible Processes in Video Generation](https://mlsanity.com/paper/2608.00617/) (2026)

## Abstract (arXiv)

Text-to-video (T2V) generation models have made rapid progress in producing visually high-quality and temporally coherent videos. However, existing benchmarks primarily focus on perceptual quality, text-video alignment, or physical plausibility, leaving a critical aspect of action understanding largely unexplored: object state change (OSC) explicitly specified in the text prompt. OSC refers to the transformation of an object's state induced by an action, such as peeling a potato or slicing a lemon. In this paper, we introduce OSCBench, a benchmark specifically designed to assess OSC performance in T2V models. OSCBench is constructed from instructional cooking data and systematically organizes action-object interactions into regular, novel, and compositional scenarios to probe both in-distribution performance and generalization. We evaluate six representative open-source and proprietary T2V models using both human user study and multimodal large language model (MLLM)-based automatic evaluation. Our results show that, despite strong performance on semantic and scene alignment, current T2V models consistently struggle with accurate and temporally consistent object state changes, especially in novel and compositional settings. These findings position OSC as a key bottleneck in text-to-video generation and establish OSCBench as a diagnostic benchmark for advancing state-aware video generation models.