Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Video event prediction (VEP) requires models to infer unobserved future states from partial video evidence. Existing video MLLMs usually verbalize intermediate future reasoning in text space: once visual evidence is verbalized, fine-grained motion, geometry, and interaction cues can be lost, leading to plausible but visually ungrounded hallucinations. We introduce Future-L1, an interleaved latent visual reasoning framework that lets an MLLM alternate between language tokens and continuous latent visual spans during autoregressive decoding. To train this capability, we construct Future-L1-50K by selecting examples where future visual hints help prediction and align latent states to future-frame embeddings, then further optimize sampled latent trajectories with LA-DAPO, a latent-aware RL objective with outcome-contrastive and temporal-diversity rewards. Future-L1 achieves new state-of-the-art results on both benchmarks: on FutureBench, it improves Qwen3-VL-8B from 61.0 to 85.4 and exceeds the previous best Video-CoE by 10.4 points; on TwiFF-Bench, it improves the average score from 2.44 to 3.04. These results suggest that future-oriented video reasoning benefits from preserving intermediate visual semantics in latent space rather than translating every reasoning step into text.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 41 of its 67 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
41 of 67 references.
Closest ancestors · 21
Same direction, newest first
- 2026HyLaR: Hybrid Latent Reasoning with Decoupled Policy Optimization22 Apr
- 2026TwiFF (Think With Future Frames): A Large-Scale Dataset for Dynamic Visual Reasoning11 Feb
- 2026SwimBird: Eliciting Switchable Reasoning Mode in Hybrid Autoregressive MLLMs5 Feb
- 2026Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning30 Jan
- 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twice8 Jan
- 2025Sketch-in-Latents: Eliciting Unified Reasoning in MLLMs18 Dec
- 2025Monet: Reasoning in Latent Visual Space Beyond Images and Language26 Nov
- 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokens24 Nov
- 2025Video-as-Answer: Predict and Generate Next Video Event with Joint-GRPO20 Nov
- 2025ExpVid: A Benchmark for Experiment Video Understanding & Reasoning13 Oct
- 2025Latent Visual Reasoning29 Sep
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
9 more
- 2025Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Models18 Aug
- 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens20 Jun
- 2025MiMo-VL Technical Report4 Jun
- 2025Fostering Video Reasoning via Next-Event Prediction28 May
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action2 May
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning9 Apr
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
- 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
Other references · 20
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
- 2026MiMo-V2-Flash Technical Reportcited by 191
- 2014Video (language) modeling: a baseline for generative models of natural videoscited by 92
- 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 70
- 2026Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understandingcited by 55
- 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
8 more
- 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
- 2026GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agentscited by 34
- 2026LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoningcited by 15
- 2026RIVER: A Real-Time Interaction Benchmark for Video LLMscited by 14
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanationcited by 10
- 2025VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMscited by 5
- 2026Video-CoE: Reinforcing Video Event Prediction via Chain of Eventscited by 4
- 2025Learning Goal-Oriented Vision-and-Language Navigation with Self-Improving Demonstrations at Scalecited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Predict, Then Retrieve: Cross-Instance Future-State Retrieval from Video Prefixes
- 2026Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
- 2025What Happens Next? Next Scene Prediction with a Unified Video Model
- 2026Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning
- 2026Latent Visual Cache for Video Reasoning
- 2026Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs
- 2026From Priors to Perception: Grounding Video-LLMs in Physical Reality
- 2026Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs