mlsanityData to 13 Oct 2026

RELIC: Interactive Video World Model with Long-Horizon Memory

arXiv 2512.04040 ↗First posted 3 Dec 2025cs.CVcited by 95

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

A truly interactive world model requires three key ingredients: real-time long-horizon streaming, consistent spatial memory, and precise user control. However, most existing approaches address only one of these aspects in isolation, as achieving all three simultaneously is highly challenging-for example, long-term memory mechanisms often degrade real-time performance. In this work, we present RELIC, a unified framework that tackles these three challenges altogether. Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time. Built upon recent autoregressive video-diffusion distillation techniques, our model represents long-horizon memory using highly compressed historical latent tokens encoded with both relative actions and absolute camera poses within the KV cache. This compact, camera-aware memory structure supports implicit 3D-consistent content retrieval and enforces long-term coherence with minimal computational overhead. In parallel, we fine-tune a bidirectional teacher video model to generate sequences beyond its original 5-second training horizon, and transform it into a causal student generator using a new memory-efficient self-forcing paradigm that enables full-context distillation over long-duration teacher as well as long student self-rollouts. Implemented as a 14B-parameter model and trained on a curated Unreal Engine-rendered dataset, RELIC achieves real-time generation at 16 FPS while demonstrating more accurate action following, more stable long-horizon streaming, and more robust spatial-memory retrieval compared with prior work. These capabilities establish RELIC as a strong foundation for the next generation of interactive world modeling.

Key facts. Cited by 95 papers in the map (data through 13 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 81 of its 88 references that are in the map. First posted on arXiv 3 Dec 2025.

Builds on

81 of 88 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
  2. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  3. 2025Yume: An Interactive World Generation Model23 Jul
  4. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling10 Jul
  5. 2025Video World Models with Long-term Spatial Memory5 Jun
  6. 2025DeepVerse: 4D Autoregressive Video Generation as a World Model1 Jun
  7. 2025Long-Context State-Space Video World Models26 May
  8. 2025Vid2World: Crafting Video Diffusion Models to Interactive World Models20 May
  9. 2025MAGI-1: Autoregressive Video Generation at Scale19 May
  10. 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
  11. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  12. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
3 more
  1. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  2. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  3. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar

Other references · 66

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,170
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,628
  3. 2023Qwen Technical Reportcited by 3,310
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,546
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,851
  6. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,775
  7. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,625
  8. 2018World Modelscited by 1,481
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,016
  10. 2023One-step Diffusion with Distribution Matching Distillationcited by 991
  11. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 664
  12. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 653
38 more
  1. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 647
  2. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 627
  3. 2024Movie Gen: A Cast of Media Foundation Modelscited by 624
  4. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 590
  5. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 589
  6. 2025Mean Flows for One-step Generative Modelingcited by 541
  7. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 515
  8. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 414
  9. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 357
  10. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 331
  11. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 301
  12. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 292
  13. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 251
  14. 2025LongLive: Real-time Interactive Long Video Generationcited by 246
  15. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 221
  16. 2025Video models are zero-shot learners and reasonerscited by 213
  17. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  18. 2025Self-Forcing++: Towards Minute-Scale High-Quality Video Generationcited by 179
  19. 2025History-Guided Video Diffusioncited by 167
  20. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 163
  21. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 159
  22. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 150
  23. 2022ARF: Artistic Radiance Fieldscited by 147
  24. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 134
  25. 2021Sequence Parallelism: Long Sequence Training from System Perspectivecited by 126
  26. 2025Test-Time Training Done Rightcited by 124
  27. 2024Rolling Diffusion Modelscited by 104
  28. 2025One-Minute Video Generation with Test-Time Trainingcited by 100
  29. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 95
  30. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 94
  31. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 92
  32. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 86
  33. 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understandingcited by 80
  34. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 79
  35. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
  36. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 72
  37. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharingcited by 71
  38. 2024Progressive Autoregressive Video Diffusion Modelscited by 65

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 4; 2026: 91.

95 papers.

Sort:

By area:

  1. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 164
  2. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 158
  3. 2026Advancing Open-source World Modelscited by 148
  4. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 125
  5. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 94
  6. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 52
  7. 2026Context Forcing: Consistent Autoregressive Video Generation with Long Contextcited by 50
  8. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 41
  9. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 39
  10. 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memorycited by 38
  11. 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 36
  12. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 32
38 more
  1. 2026LoL: Longer than Longer, Scaling Video Generation to Hourcited by 29
  2. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Modelscited by 24
  3. 2026MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokenscited by 23
  4. 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Modelscited by 23
  5. 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 23
  6. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 22
  7. 2026Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generationcited by 21
  8. 2026minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Modelscited by 19
  9. 2025TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Modelcited by 14
  10. 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learningcited by 14
  11. 2026WorldKV: Efficient World Memory with World Retrieval and Compressioncited by 14
  12. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
  13. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 13
  14. 2026LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generationcited by 13
  15. 2026ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPUcited by 13
  16. 2026Wonder: Video World Model Done Bettercited by 13
  17. 2026Alaya-EVOKE: From Linear-Scaling Supervision to Endless Worldcited by 13
  18. 2026Addressable Memory for Video World Modelscited by 12
  19. 2026Code World Model: Coding Agent as World Braincited by 11
  20. 2025LongVie 2: Multimodal Controllable Ultra-Long Video World Modelcited by 9
  21. 2026Olaf-World: Orienting Latent Actions for Video World Modelingcited by 9
  22. 2026Geometry-Aware Implicit Memory for Video World Modelscited by 9
  23. 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 9
  24. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 9
  25. 2026MemLearner: Learning to Query Context memory for Video World Modelscited by 9
  26. 2026Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memorycited by 8
  27. 2026Motus2: A Self-Evolving General World Model for Dexterous Manipulationcited by 8
  28. 2026Evolution of Video Generative Foundationscited by 7
  29. 2026Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillationcited by 7
  30. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 7
  31. 2026A Mechanistic View on Video Generation as World Models: State and Dynamicscited by 6
  32. 2026ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Modelscited by 6
  33. 2026Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generationcited by 6
  34. 2026Compression and Retrieval: Implicit Memory Retrieval for Video World Modelscited by 6
  35. 2026Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Controlcited by 6
  36. 2026ActionParty: Multi-Subject Action Binding in Generative Video Gamescited by 5
  37. 2026AAD-1: Asymmetric Adversarial Distillation for One-Step Autoregressive Video Generationcited by 5
  38. 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 5

50 most cited of 95 All 95 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Retrieve: Internalizing Memory Retrieval for Video World Models
  2. 2025MagicWorld: Towards Long-Horizon Stability for Interactive Video World Exploration
  3. 2026LIVE: Long-horizon Interactive Video World Modeling
  4. 2026The Past Frames the Future: Memory for Autoregressive Video Generation
  5. 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models
  6. 2025VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory
  7. 2026Beyond Pixel Histories: World Models with Persistent 3D State
  8. 2026AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)