# VideoVIBE: A Video-Grounded Diagnostic Benchmark for One-Shot Interactive Website Generation

arXiv: https://arxiv.org/abs/2608.09573 · HTML: https://mlsanity.com/paper/2608.09573/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 23 references that are in the map.
- First posted on arXiv 10 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe](https://mlsanity.com/paper/2509.18154/) (2025)
- [AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories](https://mlsanity.com/paper/2504.08942/) (2025)
- [Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs](https://mlsanity.com/paper/2406.20098/) (2024)
- [WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch](https://mlsanity.com/paper/2505.03733/) (2025)
- [DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generation](https://mlsanity.com/paper/2506.06251/) (2025)
- [VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks](https://mlsanity.com/paper/2410.19100/) (2024)
- [VideoGUI: A Benchmark for GUI Automation from Instructional Videos](https://mlsanity.com/paper/2406.10227/) (2024)
- [WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics](https://mlsanity.com/paper/2601.02430/) (2026)
- [Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification](https://mlsanity.com/paper/2603.26648/) (2026)
- [WebUIBench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in WebUI-to-Code](https://mlsanity.com/paper/2506.07818/) (2025)
- [Xiaomi MiMo-VL-Miloco Technical Report](https://mlsanity.com/paper/2512.17436/) (2025)
- [WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Quality](https://mlsanity.com/paper/2510.18560/) (2025)
- [Computer-Use Agents as Judges for Generative User Interface](https://mlsanity.com/paper/2511.15567/) (2025)
- [WebVR: Benchmarking Multimodal LLMs for WebPage Recreation from Videos via Human-Aligned Visual Rubrics](https://mlsanity.com/paper/2603.13391/) (2026)
- [MM-WebAgent: A Hierarchical Multimodal Web Agent for Webpage Generation](https://mlsanity.com/paper/2604.15309/) (2026)
- [WebGen-R1: Incentivizing Large Language Models to Generate Functional and Aesthetic Websites with Reinforcement Learning](https://mlsanity.com/paper/2604.20398/) (2026)
- [DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents](https://mlsanity.com/paper/2605.17439/) (2026)
- [Video2Code: Generating Interactive Webpages from UI Videos via Action-Aware Revisit](https://mlsanity.com/paper/2606.20711/) (2026)

## Built on by (most cited first, 1 of 1)

- [GameASG-Bench: Benchmarking Autonomous Software Generation for Game Development](https://mlsanity.com/paper/2609.21293/) (2026)

## Abstract (arXiv)

Natural-language-driven "vibe coding" enables the one-shot generation of visually rich and interactive web applications, yet reliable assessment of their quality has not kept pace. Existing evaluations often score isolated artifacts or final task outcomes, offering limited evidence about which failures occur and why. We introduce VideoVIBE, a video-grounded benchmark that transforms human-operated webpage recordings into fine-grained diagnostic tasks. It contains approximately 1.7K diagnostic Video QA instances derived from 6,338 verified failures across generated webpages, spanning semantic-logical, visual-motion, structural-temporal, and functional failures. Diagnoses are grounded primarily in recorded presentation and behavior, with webpage source code used as complementary context. We further propose V2Lens, a training-free, evidence-grounded multi-agent system that challenges and selectively refines initial video-based diagnoses through targeted visual and source-code verification. Across thirteen closed-source and open-weight Video MLLMs, Gemini-2.5-Flash is the strongest standalone model with a score of 64.54, while V2Lens reaches 71.72, an improvement of 7.18 points. Together, our results show that video-grounded evaluation can move beyond isolated artifacts and aggregate outcomes toward a behaviorally faithful and diagnostically informative account of generated application quality.