mlsanityData to 14 Oct 2026

DDLP: Unsupervised Object-Centric Video Prediction with Deep Dynamic Latent Particles

arXiv 2306.05957 ↗First posted 9 Jun 2023cs.CV cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

We propose a new object-centric video prediction algorithm based on the deep latent particle (DLP) representation. In comparison to existing slot- or patch-based representations, DLPs model the scene using a set of keypoints with learned parameters for properties such as position and size, and are both efficient and interpretable. Our method, deep dynamic latent particles (DDLP), yields state-of-the-art object-centric video prediction results on several challenging datasets. The interpretable nature of DDLP allows us to perform ``what-if'' generation -- predict the consequence of changing properties of objects in the initial frames, and DLP's compact structure enables efficient diffusion-based unconditional video generation. Videos, code and pre-trained models are available: https://taldatech.github.io/ddlp-web

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 62 of its 81 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

62 of 81 references.

References · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  9. 2015Spatial Transformer Networkscited by 1,429
  10. 2018Self-Attention with Relative Position Representationscited by 1,120
  11. 2020Object-Centric Learning with Slot Attentioncited by 876
  12. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
38 more
  1. 2017The Perception-Distortion Tradeoffcited by 587
  2. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 509
  3. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
  4. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  5. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  6. 2022MAGVIT: Masked Generative Video Transformercited by 346
  7. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 260
  8. 2021Conditional Object-Centric Learning from Videocited by 248
  9. 2018Stochastic Video Generation with a Learned Priorcited by 245
  10. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  11. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  12. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  13. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 182
  14. 2018Stochastic Adversarial Video Predictioncited by 173
  15. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 166
  16. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 153
  17. 2021Illiterate DALL-E Learns to Composecited by 147
  18. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
  19. 2017Self-Supervised Visual Planning with Temporal Skip Connectionscited by 139
  20. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  21. 2019PHYRE: A New Benchmark for Physical Reasoningcited by 137
  22. 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 114
  23. 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
  24. 2019Unsupervised Learning of Object Keypoints for Perception and Controlcited by 109
  25. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  26. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  27. 2019Unsupervised Learning of Object Structure and Dynamics from Videoscited by 92
  28. 2022Object Scene Representation Transformercited by 90
  29. 2016Learning to Track at 100 FPS with Deep Regression Networkscited by 89
  30. 2017Learning to Generate Long-term Future via Hierarchical Predictioncited by 88
  31. 2019High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networkscited by 88
  32. 2020Causal Discovery in Physical Systems from Videoscited by 77
  33. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  34. 2020Improving Generative Imagination in Object-Centric World Modelscited by 71
  35. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  36. 2021Predicting Video with VQVAEcited by 61
  37. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  38. 2019Structured Object-Aware Physics Prediction for Video Modeling and Planningcited by 43

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 6; 2026: 7.

19 papers.

Sort:

By area:

  1. 2024Entity-Centric Reinforcement Learning for Object Manipulation from Pixelscited by 27
  2. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 15
  3. 2025PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planningcited by 14
  4. 2024Unsupervised Discovery of Object-Centric Neural Fieldscited by 12
  5. 2023Benchmarks for Physical Reasoning AIcited by 8
  6. 2024EC-Diffuser: Multi-Object Manipulation via Entity-Centric Behavior Generationcited by 8
  7. 2026Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusioncited by 7
  8. 2023Relational Object-Centric Actor-Criticcited by 4
  9. 2024OCK: Unsupervised Dynamic Video Prediction with Object-Centric Kinematicscited by 3
  10. 2025TextOCVP: Object-Centric Video Prediction with Language Guidancecited by 1
  11. 2025Enhancing Temporal Awareness in LLMs for Temporal Point Processescited by 1
  12. 20263D-DLP: Self-Supervised 3D Object-Centric Scene Representation Learningcited by 1
7 more
  1. 2025VideoPCDNet: Video Parsing and Prediction with Phase Correlation Networkscited by 0
  2. 2025Object-centric Denoising Diffusion Models for Physical Reasoningcited by 0
  3. 2025When Object-Centric World Models Meet Policy Learning: From Pixels to Policies, and Where It Breakscited by 0
  4. 2026More with LESS -- Local Scene Representations for Tactile Imagingcited by 0
  5. 2026ParticleSplat: Self-supervised Object-centric Latent Particle Splattingcited by 0
  6. 2026iSEE: Object Permanence Through Self-Supervisioncited by 0
  7. 2026VCLMU: Mechanism-Centric Virtual Cell World Modeling for Perturbation Responsecited by 0