mlsanityData to 7 Oct 2026

What Happens Next? Next Scene Prediction with a Unified Video Model

arXiv 2512.13015 ↗First posted 15 Dec 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent unified models for joint understanding and generation have significantly advanced visual generation capabilities. However, their focus on conventional tasks like text-to-video generation has left the temporal reasoning potential of unified models largely underexplored. To address this gap, we introduce Next Scene Prediction (NSP), a new task that pushes unified video models toward temporal and causal reasoning. Unlike text-to-video generation, NSP requires predicting plausible futures from preceding context, demanding deeper understanding and reasoning. To tackle this task, we propose a unified framework combining Qwen-VL for comprehension and LTX for synthesis, bridged by a latent query embedding and a connector module. This model is trained in three stages on our newly curated, large-scale NSP dataset: text-to-video pre-training, supervised fine-tuning, and reinforcement learning (via GRPO) with our proposed causal consistency reward. Experiments demonstrate our model achieves state-of-the-art performance on our benchmark, advancing the capability of generalist multimodal systems to anticipate what happens next.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 60 of its 63 references that are in the map. First posted on arXiv 15 Dec 2025.

Builds on

60 of 63 references.

References · 60

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2022Flow Matching for Generative Modelingcited by 5,426
  7. 2025Qwen2.5-VL Technical Reportcited by 5,418
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  10. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  11. 2019Root Mean Square Layer Normalizationcited by 1,425
  12. 2023Training Diffusion Models with Reinforcement Learningcited by 939
38 more
  1. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  2. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  3. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  4. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  5. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  6. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  7. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  8. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  9. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  10. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  11. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  12. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  13. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
  14. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  15. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
  16. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  17. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  18. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 246
  19. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  20. 2025Transfer between Modalities with MetaQueriescited by 231
  21. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
  22. 2025HunyuanImage 3.0 Technical Reportcited by 138
  23. 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 129
  24. 2025Long Context Tuning for Video Generationcited by 110
  25. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
  26. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
  27. 2024Video Diffusion Alignment via Reward Gradientscited by 101
  28. 2024Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenizationcited by 98
  29. 2024Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generationcited by 85
  30. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  31. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  32. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  33. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  34. 2024VidGen-1M: A Large-Scale Dataset for Text-to-video Generationcited by 60
  35. 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 45
  36. 2025OpenUni: A Simple Baseline for Unified Multimodal Understanding and Generationcited by 45
  37. 2025Omni-Video: Democratizing Unified Video Understanding and Generationcited by 39
  38. 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimizationcited by 38

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
  2. 2025Fostering Video Reasoning via Next-Event Prediction
  3. 2025Video-as-Answer: Predict and Generate Next Video Event with Joint-GRPO
  4. 2026Representations Before Pixels: Semantics-Guided Hierarchical Video Prediction
  5. 2026VepAgent: Bridging Causal-Transition via Tool-Augmented Reinforcement Learning for Video Event Prediction
  6. 2026ProAR: Learning Prospective Reasoning with Autoregressive Video Models
  7. 2025Learning from Next-Frame Prediction: Autoregressive Video Modeling Encodes Effective Representations
  8. 2026X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling