# Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft

arXiv: https://arxiv.org/abs/2510.03198 · HTML: https://mlsanity.com/paper/2510.03198/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 42 of its 42 references that are in the map.
- First posted on arXiv 3 Oct 2025.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)
- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 42)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass](https://mlsanity.com/paper/2501.13928/) (2025)
- [MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation](https://mlsanity.com/paper/2205.09853/) (2022)
- [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024)
- [Flexible Diffusion Modeling of Long Videos](https://mlsanity.com/paper/2205.11495/) (2022)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [Navigation World Models](https://mlsanity.com/paper/2412.03572/) (2024)
- [LongLive: Real-time Interactive Long Video Generation](https://mlsanity.com/paper/2509.22622/) (2025)
- [StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text](https://mlsanity.com/paper/2403.14773/) (2024)
- [MineRL: A Large-Scale Dataset of Minecraft Demonstrations](https://mlsanity.com/paper/1907.13440/) (2019)
- [Matrix-game 2.0: An open-source real-time and streaming interactive world model](https://mlsanity.com/paper/2508.13009/) (2025)
- [History-Guided Video Diffusion](https://mlsanity.com/paper/2502.06764/) (2025)
- [WorldMem: Long-term Consistent World Simulation with Memory](https://mlsanity.com/paper/2504.12369/) (2025)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [iVideoGPT: Interactive VideoGPTs are Scalable World Models](https://mlsanity.com/paper/2405.15223/) (2024)
- [VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold](https://mlsanity.com/paper/2505.12549/) (2025)
- [GameFactory: Creating New Games with Generative Interactive Videos](https://mlsanity.com/paper/2501.08325/) (2025)

## Built on by (most cited first, 30 of 52)

- [Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model](https://mlsanity.com/paper/2511.23429/) (2025)
- [StoryMem: Multi-shot Long Video Storytelling with Memory](https://mlsanity.com/paper/2512.19539/) (2025)
- [Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory](https://mlsanity.com/paper/2602.02393/) (2026)
- [MultiShotMaster: A Controllable Multi-Shot Video Generation Framework](https://mlsanity.com/paper/2512.03041/) (2025)
- [MIND: Benchmarking Memory Consistency and Action Control in World Models](https://mlsanity.com/paper/2602.08025/) (2026)
- [Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models](https://mlsanity.com/paper/2603.25716/) (2026)
- [Memorize-and-Generate: Towards Long-Term Consistency in Real-Time Video Generation](https://mlsanity.com/paper/2512.18741/) (2025)
- [LIVE: Long-horizon Interactive Video World Modeling](https://mlsanity.com/paper/2602.03747/) (2026)
- [MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines](https://mlsanity.com/paper/2603.06679/) (2026)
- [Pathwise Test-Time Correction for Autoregressive Long Video Generation](https://mlsanity.com/paper/2602.05871/) (2026)
- [WorldKV: Efficient World Memory with World Retrieval and Compression](https://mlsanity.com/paper/2605.22718/) (2026)
- [Wonder: Video World Model Done Better](https://mlsanity.com/paper/2607.26037/) (2026)
- [Beyond Pixel Histories: World Models with Persistent 3D State](https://mlsanity.com/paper/2603.03482/) (2026)
- [Geometry-Aware Implicit Memory for Video World Models](https://mlsanity.com/paper/2606.02436/) (2026)
- [Geometry-Aware Rotary Position Embedding for Consistent Video World Model](https://mlsanity.com/paper/2602.07854/) (2026)
- [Mode Seeking meets Mean Seeking for Fast Long Video Generation](https://mlsanity.com/paper/2602.24289/) (2026)
- [SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models](https://mlsanity.com/paper/2609.02886/) (2026)
- [Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends](https://mlsanity.com/paper/2606.01164/) (2026)
- [Efficient Video Diffusion Models: Advancements and Challenges](https://mlsanity.com/paper/2604.15911/) (2026)
- [Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generation](https://mlsanity.com/paper/2604.18215/) (2026)
- [Echo-Memory: A Controlled Study of Memory in Action World Models](https://mlsanity.com/paper/2606.09803/) (2026)
- [WorldOlympiad: Can Your World Model Survive a Triathlon?](https://mlsanity.com/paper/2606.11129/) (2026)
- [Vision-Language Memory for Spatial Reasoning](https://mlsanity.com/paper/2511.20644/) (2025)
- [Captain Safari: A World Engine with Pose-Aligned 3D Memory](https://mlsanity.com/paper/2511.22815/) (2025)
- [Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms](https://mlsanity.com/paper/2603.28489/) (2026)
- [Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM](https://mlsanity.com/paper/2606.26120/) (2026)
- [Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion](https://mlsanity.com/paper/2608.26794/) (2026)
- [Efficient Reasoning with Balanced Thinking](https://mlsanity.com/paper/2603.12372/) (2026)
- [MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation](https://mlsanity.com/paper/2606.09056/) (2026)
- [GeoStream: Toward Precise Camera Controlled Streaming Video Generation](https://mlsanity.com/paper/2606.15162/) (2026)

## Abstract (arXiv)

Autoregressive video diffusion models have proved effective for world modeling and interactive scene generation, with Minecraft gameplay as a representative application. To faithfully simulate play, a model must generate natural content while exploring new scenes and preserve spatial consistency when revisiting explored areas. Under limited computation budgets, it must compress and exploit historical cues within a finite context window, which exposes a trade-off: Temporal-only memory lacks long-term spatial consistency, whereas adding spatial memory strengthens consistency but may degrade new scene generation quality when the model over-relies on insufficient spatial context. We present Memory Forcing, a learning framework that pairs training protocols with a geometry-indexed spatial memory. Hybrid Training exposes distinct gameplay regimes, guiding the model to rely on temporal memory during exploration and incorporate spatial memory for revisits. Chained Forward Training extends autoregressive training with model rollouts, where chained predictions create larger pose variations and encourage reliance on spatial memory for maintaining consistency. Point-to-Frame Retrieval efficiently retrieves history by mapping currently visible points to their source frames, while Incremental 3D Reconstruction maintains and updates an explicit 3D cache. Extensive experiments demonstrate that Memory Forcing achieves superior long-term spatial consistency and generative quality across diverse environments, while maintaining computational efficiency for extended sequences.