mlsanityData to 7 Oct 2026

Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling

arXiv 2603.04553 ↗First posted 4 Mar 2026cs.LGcited by 11

On the map: World models → Latent predictive world models

Abstract (arXiv)

We introduce Latent Particle World Model (LPWM), a self-supervised object-centric world model scaled to real-world multi-object datasets and applicable in decision-making. LPWM autonomously discovers keypoints, bounding boxes, and object masks directly from video data, enabling it to learn rich scene decompositions without supervision. Our architecture is trained end-to-end purely from videos and supports flexible conditioning on actions, language, and image goals. LPWM models stochastic particle dynamics via a novel latent action module and achieves state-of-the-art results on diverse real-world and synthetic datasets. Beyond stochastic video modeling, LPWM is readily applicable to decision-making, including goal-conditioned imitation learning, as we demonstrate in the paper. Code, data, pre-trained models and video rollouts are available: https://taldatech.github.io/lpwm-web

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 132 of its 158 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

132 of 158 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Dyn-O: Building Structured World Models with Object-Centric Representations4 Jul
  2. 2021Conditional Object-Centric Learning from Video24 Nov
  3. 2019SCALOR: Generative World Models with Scalable Object Representations6 Oct

Other references · 129

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2022Scalable Diffusion Models with Transformerscited by 5,931
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2022Video Diffusion Modelscited by 2,280
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  12. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
38 more
  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  3. 2018World Modelscited by 1,444
  4. 2015Spatial Transformer Networkscited by 1,423
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  7. 2023Mastering Diverse Domains through World Modelscited by 1,193
  8. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  9. 2020Mastering Atari with Discrete World Modelscited by 990
  10. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  11. 2020Object-Centric Learning with Slot Attentioncited by 865
  12. 2024Genie: Generative Interactive Environmentscited by 706
  13. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  14. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  15. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  16. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  17. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 475
  18. 2023Learning Interactive Real-World Simulatorscited by 440
  19. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  20. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  21. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  22. 2022MAGVIT: Masked Generative Video Transformercited by 341
  23. 2022Transformers are Sample-Efficient World Modelscited by 317
  24. 2019Learning Latent Plans from Playcited by 310
  25. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  26. 2024Latent Action Pretraining from Videoscited by 302
  27. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  28. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  29. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  30. 2021N"UWA: Visual Synthesis Pre-training for Neural visUal World creAtioncited by 253
  31. 2018Stochastic Video Generation with a Learned Priorcited by 244
  32. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  33. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  34. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  35. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  36. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
  37. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  38. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181

50 most cited of 129 All 129 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026Latent State Design for World Models under Sufficiency Constraintscited by 3
  2. 2026Unifying Object-Centric World Models and Diffusion Policy: A Hierarchical Framework for Multi-Stage Robotic Taskscited by 2
  3. 20263D-DLP: Self-Supervised 3D Object-Centric Scene Representation Learningcited by 1
  4. 2026Latent Goal Prediction from Language for Model-Based Planningcited by 1
  5. 2026PhysCoRe: Physics-Corrected Residual World Models for Material-Aware Deformable Dynamicscited by 1
  6. 2026Render, Don't Decode: Weight-Space World Models with Latent Structural Disentanglementcited by 0
  7. 2026Latent Actions from Factorized Transition Effects under Agent Ambiguitycited by 0
  8. 2026SlotDiT: Object-Centric Representations for Diffusion Transformerscited by 0
  9. 2026ParticleSplat: Self-supervised Object-centric Latent Particle Splattingcited by 0
  10. 2026WorldGraph: Graph-Native World Modelingcited by 0
  11. 2026iSEE: Object Permanence Through Self-Supervisioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models
  2. 2026Learning Latent Action World Models In The Wild
  3. 2026Learning a Particle Dynamics Model with Real-world Videos
  4. 2026LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations
  5. 2026Factored Latent Action World Models
  6. 2026StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions
  7. 2026Learning How the World Evolves: Extrapolative Video World Models via Latent Dynamics Reasoning
  8. 2026Causally Debiased Latent Action Model for Embodied Action Conditioned World Models