mlsanityData to 14 Oct 2026

Improving Generative Imagination in Object-Centric World Models

arXiv 2010.02054 ↗First posted 5 Oct 2020cs.LG stat.MLcited by 71

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

The remarkable recent advances in object-centric generative world models raise a few questions. First, while many of the recent achievements are indispensable for making a general and versatile world model, it is quite unclear how these ingredients can be integrated into a unified framework. Second, despite using generative objectives, abilities for object detection and tracking are mainly investigated, leaving the crucial ability of temporal imagination largely under question. Third, a few key abilities for more faithful temporal imagination such as multimodal uncertainty and situation-awareness are missing. In this paper, we introduce Generative Structured World Models (G-SWM). The G-SWM achieves the versatile world modeling not only by unifying the key properties of previous models in a principled framework but also by achieving two crucial new abilities, multimodal uncertainty and situation-awareness. Our thorough investigation on the temporal generation ability in comparison to the previous models demonstrates that G-SWM achieves the versatility with the best or comparable performance for all experiment settings including a few complex settings that have not been tested before.

Key facts. Cited by 71 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 2 of its 50 references that are in the map. First posted on arXiv 5 Oct 2020.

Builds on

2 of 50 references.

References · 2

  1. 2018World Modelscited by 1,513
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 14; 2023: 15; 2024: 12; 2025: 10; 2026: 10.

71 papers.

Sort:

By area:

  1. 2021Illiterate DALL-E Learns to Composecited by 147
  2. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
  3. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  4. 2023Object-Centric Slot Diffusioncited by 90
  5. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  6. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 85
  7. 2021Generalization and Robustness Implications in Object-Centric Learningcited by 81
  8. 2022Learning Multi-Object Dynamics with Compositional Neural Radiance Fieldscited by 63
  9. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  10. 2022Neural Systematic Bindercited by 37
  11. 2021Unsupervised Object Learning via Common Fatecited by 29
  12. 2022Compositional Scene Representation Learning via Reconstruction: A Surveycited by 27
38 more
  1. 2021Generative Video Transformer: Can Objects be the Words?cited by 24
  2. 2022Toward Compositional Generalization in Object-Oriented World Modelingcited by 22
  3. 2021Structured World Belief for Reinforcement Learning in POMDPcited by 21
  4. 2022Unsupervised Multi-object Segmentation by Predicting Probable Motion Patternscited by 19
  5. 2023Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactionscited by 19
  6. 2023DDLP: Unsupervised Object-Centric Video Prediction with Deep Dynamic Latent Particlescited by 19
  7. 2024Data-Centric Evolution in Autonomous Driving: A Comprehensive Survey of Big Data System, Data Mining, and Closed-Loop Technologiescited by 17
  8. 2024Slot State Space Modelscited by 17
  9. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 15
  10. 2023Neuro-Symbolic World Models for Adapting to Open World Noveltycited by 14
  11. 2024Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Modelscited by 14
  12. 2021Reasoning-Modulated Representationscited by 13
  13. 2022Inductive Biases for Object-Centric Representations in the Presence of Complex Texturescited by 13
  14. 2022TarGF: Learning Target Gradient Field to Rearrange Objects without Explicit Goal Specificationcited by 13
  15. 2025Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generationcited by 13
  16. 2021Object-Centric Representation Learning with Generative Spatial-Temporal Factorizationcited by 10
  17. 2025Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learningcited by 10
  18. 2025DMWM: Dual-Mind World Model with Long-Term Imaginationcited by 10
  19. 2025Four Principles for Physically Interpretable World Modelscited by 10
  20. 2025Direct Motion Models for Assessing Generated Videoscited by 10
  21. 2026A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokenscited by 10
  22. 2024Neural Language of Thought Modelscited by 9
  23. 2021GATSBI: Generative Agent-centric Spatio-temporal Object Interactioncited by 8
  24. 2023Controllable Video Generation by Learning the Underlying Dynamical System with Neural ODEcited by 7
  25. 2024Reasoning-Enhanced Object-Centric Learning for Videoscited by 7
  26. 2021Unsupervised Video Prediction from a Single Frame by Estimating 3D Dynamic Scene Structurecited by 6
  27. 2023Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Modelscited by 6
  28. 2026Graph World Models: Concepts, Taxonomy, and Future Directionscited by 6
  29. 2024Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biasescited by 5
  30. 2023Reusable Slotwise Mechanismscited by 4
  31. 2023Learning Object Permanence from Videos via Latent Imaginationscited by 4
  32. 2024Compete and Compose: Learning Independent Mechanisms for Modular World Modelscited by 4
  33. 2024FACTS: A Factored State-Space Framework For World Modellingcited by 4
  34. 2025Dreamweaver: Learning Compositional World Models from Pixelscited by 4
  35. 2025Motion-Refined DINOSAUR for Unsupervised Multi-Object Discoverycited by 4
  36. 2022Patch-based Object-centric Transformers for Efficient Video Generationcited by 3
  37. 2022Compositional Law Parsing with Latent Random Functionscited by 3
  38. 2023Time-Conditioned Generative Modeling of Object-Centric Representations for Video Decomposition and Predictioncited by 3

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026WorldCast: Exploring Intelligent Visual Generation in World State Prediction
  2. 2025Can World Models Benefit VLMs for World Dynamics?
  3. 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistency
  4. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Models
  5. 2025From Masks to Worlds: A Hitchhiker's Guide to World Models
  6. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models
  7. 2025WorldModelBench: Judging Video Generation Models As World Models
  8. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens