What Happens Next? Next Scene Prediction with a Unified Video Model
Not on the map. Why?
Abstract (arXiv)
Recent unified models for joint understanding and generation have significantly advanced visual generation capabilities. However, their focus on conventional tasks like text-to-video generation has left the temporal reasoning potential of unified models largely underexplored. To address this gap, we introduce Next Scene Prediction (NSP), a new task that pushes unified video models toward temporal and causal reasoning. Unlike text-to-video generation, NSP requires predicting plausible futures from preceding context, demanding deeper understanding and reasoning. To tackle this task, we propose a unified framework combining Qwen-VL for comprehension and LTX for synthesis, bridged by a latent query embedding and a connector module. This model is trained in three stages on our newly curated, large-scale NSP dataset: text-to-video pre-training, supervised fine-tuning, and reinforcement learning (via GRPO) with our proposed causal consistency reward. Experiments demonstrate our model achieves state-of-the-art performance on our benchmark, advancing the capability of generalist multimodal systems to anticipate what happens next.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 60 of its 63 references that are in the map. First posted on arXiv 15 Dec 2025.
Builds on
60 of 63 references.
References · 60
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2019Root Mean Square Layer Normalizationcited by 1,425
- 2023Training Diffusion Models with Reinforcement Learningcited by 939
38 more
- 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
- 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
- 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
- 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
- 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 246
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2025Transfer between Modalities with MetaQueriescited by 231
- 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
- 2025HunyuanImage 3.0 Technical Reportcited by 138
- 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 129
- 2025Long Context Tuning for Video Generationcited by 110
- 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
- 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
- 2024Video Diffusion Alignment via Reward Gradientscited by 101
- 2024Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenizationcited by 98
- 2024Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generationcited by 85
- 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
- 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
- 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
- 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
- 2024VidGen-1M: A Large-Scale Dataset for Text-to-video Generationcited by 60
- 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 45
- 2025OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generationcited by 45
- 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 39
- 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimizationcited by 38
50 most cited of 60 All 60 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026Advances in GRPO for Generation Models: A Surveycited by 1
- 2026LogiShot: Logically Coherent Cross-Shot Video Generationcited by 1
- 2026LogiShot: Logically Coherent Cross-Shot Video Generation9 Aug 2026
- 2026Advances in GRPO for Generation Models: A Survey21 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
- 2025Fostering Video Reasoning via Next-Event Prediction
- 2025Video-as-Answer: Predict and Generate Next Video Event with Joint-GRPO
- 2026Representations Before Pixels: Semantics-Guided Hierarchical Video Prediction
- 2026VepAgent: Bridging Causal-Transition via Tool-Augmented Reinforcement Learning for Video Event Prediction
- 2026ProAR: Learning Prospective Reasoning with Autoregressive Video Models
- 2025Learning from Next-Frame Prediction: Autoregressive Video Modeling Encodes Effective Representations
- 2026X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling