mlsanityData to 14 Oct 2026

Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution

arXiv 2605.25333 ↗First posted 25 May 2026cs.CVcited by 4

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Video world models should maintain evolving states when evidence is unobserved, yet current generators often freeze hidden states upon interruption. This is not simply a capacity problem: pretrained video diffusion transformers already possess KV-cache mechanisms capable of non-local retrieval, but they are rarely trained to use them as dynamic memory. We introduce ReMind, a framework eliciting dynamic memory behavior via memory-oriented data, event-aware training, and cache adaptation. Organized around a taxonomy of 100+ dynamic events, we build a camera-annotated training mixture combining VLM-filtered real videos, generated hard dynamics, synthetic camera loops, and memory-interruption augmentations. Each clip is converted into a frame graph with protected anchors, degraded intervals, and explicit temporal gaps. A node-structured curriculum -- including node-drop, noisy memory, frontier continuation, and reference-cache training -- forces the model to retrieve relevant past states across interruptions rather than relying solely on local continuity. PM-RoPE, an elegant camera-phase RoPE extension, unlocks spatiotemporal retrieval at a single-attention cost while preserving pretrained pathways. ReMind achieves the best overall scores on STEVO-Bench and recovery tasks. Furthermore, general image-to-video evaluations confirm this curriculum avoids catastrophic forgetting. We have released our code, data, and models on our project page hrefhttps://remind-applied.github.io/https://remind-applied.github.io/.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 28 of its 30 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

28 of 30 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling8 Apr
  2. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Models17 Mar
  3. 2026Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models13 Mar
  4. 2026RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space24 Feb
  5. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos1 Jan
  6. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  7. 2025Unified Camera Positional Encoding for Controlled Video Generation8 Dec
  8. 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar

Other references · 19

  1. 2025Qwen3-VL Technical Reportcited by 2,793
  2. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  3. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  4. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  5. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  6. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  7. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  8. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  9. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  10. 2025LongLive: Real-time Interactive Long Video Generationcited by 265
  11. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 163
  12. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
7 more
  1. 2025Cameras as Relative Positional Encodingcited by 105
  2. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
  3. 2026ReRoPE: Repurposing RoPE for Relative Camera Controlcited by 13
  4. 2026URoPE: Universal Relative Position Embedding across Geometric Spacescited by 6
  5. 2025STANCE: Motion Coherent Video Generation Via Sparse-to-Dense Anchored Encodingcited by 5
  6. 2026Motion Forcing: A Decoupled Framework for Robust Video Generation in Motion Dynamicscited by 3
  7. 2026KV Cache Quantization for Self-Forcing Video Generation: A 33-Method Empirical Studycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Memory Forcing: Attendable Mid-Horizon History for Streaming Video Generation
  2. 2026LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation
  3. 2026The Past Frames the Future: Memory for Autoregressive Video Generation
  4. 2026Echo-Memory: A Controlled Study of Memory in Action World Models
  5. 2026Compress to Remember: Learning Compact Memory via On-Policy Distillation for Long Video Generation
  6. 2025Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generation
  7. 2026Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion
  8. 2026Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation