mlsanityData to 7 Oct 2026

Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactions

arXiv 2302.11850 ↗First posted 23 Feb 2023cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

We propose a novel framework for the task of object-centric video prediction, i.e., extracting the compositional structure of a video sequence, as well as modeling objects dynamics and interactions from visual observations in order to predict the future object states, from which we can then generate subsequent video frames. With the goal of learning meaningful spatio-temporal object representations and accurately forecasting object states, we propose two novel object-centric video predictor (OCVP) transformer modules, which decouple the processing of temporal dynamics and object interactions, thus presenting an improved prediction performance. In our experiments, we show how our object-centric prediction framework utilizing our OCVP predictors outperforms object-agnostic video prediction models on two different datasets, while maintaining consistent and accurate object representations.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 26 of its 29 references that are in the map. First posted on arXiv 23 Feb 2023.

Builds on

26 of 29 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  4. 2020Object-Centric Learning with Slot Attentioncited by 865
  5. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  6. 2021Conditional Object-Centric Learning from Videocited by 247
  7. 2018Stochastic Video Generation with a Learned Priorcited by 244
  8. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  9. 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 192
  10. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  11. 2019Scaling Autoregressive Video Modelscited by 189
  12. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
14 more
  1. 2022SimVP: Simpler yet Better Video Predictioncited by 181
  2. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 146
  3. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  4. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  5. 2020Latent Video Transformercited by 106
  6. 2020Improving Generative Imagination in Object-Centric World Modelscited by 70
  7. 2019Compositional Video Predictioncited by 43
  8. 2021Generative Video Transformer: Can Objects be the Words?cited by 24
  9. 2018Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackerscited by 22
  10. 2019Segmenting the Futurecited by 19
  11. 2021Unsupervised Object-Based Transition Models for 3D Partially Observable Environmentscited by 19
  12. 2022MSPred: Video Prediction at Multiple Spatio-Temporal Scales with Hierarchical Recurrent Networkscited by 12
  13. 2021Local Frequency Domain Transformer Networks for Video Predictioncited by 9
  14. 2021Unsupervised Image Decomposition with Phase-Correlation Networkscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 5; 2026: 7.

19 papers.

Sort:

By area:

  1. 2026Causal-JEPA: Learning World Models through Object-Level Latent Maskingcited by 25
  2. 2023DDLP: Unsupervised Object-Centric Video Prediction with Deep Dynamic Latent Particlescited by 18
  3. 2025PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planningcited by 13
  4. 2024SLCF-Net: Sequential LiDAR-Camera Fusion for Semantic Scene Completion using a 3D Recurrent U-Netcited by 12
  5. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 11
  6. 2024A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approachescited by 7
  7. 2024Reasoning-Enhanced Object-Centric Learning for Videoscited by 7
  8. 2025Smoothing Slot Attention Iterations and Recurrencescited by 5
  9. 2024OCK: Unsupervised Dynamic Video Prediction with Object-Centric Kinematicscited by 3
  10. 2024Top-Down Guidance for Learning Object-Centric Representationscited by 3
  11. 2026Space-Time Forecasting of Dynamic Scenes with Motion-aware Gaussian Groupingcited by 3
  12. 2024Object-centric proto-symbolic behavioural reasoning from pixelscited by 2
7 more
  1. 2025TextOCVP: Object-Centric Video Prediction with Language Guidancecited by 1
  2. 2026Rethinking Object-Centric Representations for Video Dynamics Modelingcited by 1
  3. 2025OC-SOP: Enhancing Vision-Based 3D Semantic Occupancy Prediction by Object-Centric Awarenesscited by 0
  4. 2025VideoPCDNet: Video Parsing and Prediction with Phase Correlation Networkscited by 0
  5. 2026Event-Conditioned Diagnostics of Kinematic, Contact, and Object-Permanence Structure in Passive Object-State World Modelscited by 0
  6. 2026SlotDiT: Object-Centric Representations for Diffusion Transformerscited by 0
  7. 2026iSEE: Object Permanence Through Self-Supervisioncited by 0