mlsanityData to 7 Oct 2026

Unsupervised object-centric video generation and decomposition in 3D

arXiv 2007.06705 ↗First posted 7 Jul 2020cs.CV cs.LG stat.MLcited by 31

Not on the map. Why?

Abstract (arXiv)

A natural approach to generative modeling of videos is to represent them as a composition of moving objects. Recent works model a set of 2D sprites over a slowly-varying background, but without considering the underlying 3D scene that gives rise to them. We instead propose to model a video as the view seen while moving through a scene with multiple 3D objects and a 3D background. Our model is trained from monocular videos without any supervision, yet learns to generate coherent 3D scenes containing several moving objects. We conduct detailed experiments on two datasets, going beyond the visual complexity supported by state-of-the-art generative approaches. We evaluate our method on depth-prediction and 3D object detection -- tasks which cannot be addressed by those earlier works -- and show it out-performs them even on 2D instance segmentation and tracking.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 7; 2023: 7; 2024: 6; 2025: 1; 2026: 2.

31 papers.

Sort:

By area:

  1. 2021Conditional Object-Centric Learning from Videocited by 247
  2. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  3. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 111
  4. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  5. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  6. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  7. 2022Discovering Objects that Can Movecited by 44
  8. 2022Neural Systematic Bindercited by 37
  9. 2023PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervisioncited by 36
  10. 2023Helping Hands: An Object-Aware Ego-Centric Video Recognition Modelcited by 31
  11. 2023Object Discovery from Motion-Guided Tokenscited by 25
  12. 2022Is an Object-Centric Video Representation Beneficial for Transfer?cited by 24
19 more
  1. 2023CAD: Photorealistic 3D Generation via Adversarial Distillationcited by 18
  2. 20223D-Aware Video Generationcited by 17
  3. 2023Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learningcited by 17
  4. 2024Denoising Diffusion via Image-Based Renderingcited by 15
  5. 2024Unsupervised Discovery of Object-Centric Neural Fieldscited by 12
  6. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Modelscited by 10
  7. 2021Autoencoder-based background reconstruction and foreground segmentation with background noise estimationcited by 7
  8. 2024Reasoning-Enhanced Object-Centric Learning for Videoscited by 7
  9. 2021Unsupervised Video Prediction from a Single Frame by Estimating 3D Dynamic Scene Structurecited by 6
  10. 2022WALDO: Future Video Synthesis using Object Layer Decomposition and Parametric Flow Predictioncited by 5
  11. 2025Counterfactual World Models via Digital Twin-conditioned Video Diffusioncited by 4
  12. 2021Spatially Invariant Unsupervised 3D Object-Centric Learning and Scene Decompositioncited by 2
  13. 2024Parallelized Spatiotemporal Bindingcited by 2
  14. 2021InSeGAN: A Generative Approach to Segmenting Identical Instances in Depth Imagescited by 1
  15. 2022V3GAN: Decomposing Background, Foreground and Motion for Video Generationcited by 1
  16. 2026Pixel-to-4D: Camera-Controlled Image-to-Video Generation with Dynamic 3D Gaussianscited by 1
  17. 2023Variational Inference for Scalable 3D Object-centric Learningcited by 0
  18. 2024Learning 3D object-centric representation through predictioncited by 0
  19. 2026Towards Stable Self-Supervised Object Representations in Unconstrained Egocentric Videocited by 0