# Weakly Supervised Learning of Multi-Object 3D Scene Decompositions Using Deep Shape Priors

arXiv: https://arxiv.org/abs/2010.04030 · HTML: https://mlsanity.com/paper/2010.04030/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [Learning Multi-Object Dynamics with Compositional Neural Radiance Fields](https://mlsanity.com/paper/2202.11855/) (2022)
- [Neural Groundplans: Persistent Neural Scene Representations from a Single Image](https://mlsanity.com/paper/2207.11232/) (2022)
- [Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learning](https://mlsanity.com/paper/2303.17842/) (2023)
- [GLASS: Guided Latent Slot Diffusion for Object-Centric Learning](https://mlsanity.com/paper/2407.17929/) (2024)
- [SSDNeRF: Semantic Soft Decomposition of Neural Radiance Fields](https://mlsanity.com/paper/2212.03406/) (2022)
- [Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior](https://mlsanity.com/paper/2305.19550/) (2023)
- [ObPose: Leveraging Pose for Object-Centric Scene Inference and Generation in 3D](https://mlsanity.com/paper/2206.03591/) (2022)
- [Disentangling Visual Priors: Unsupervised Learning of Scene Interpretations with Compositional Autoencoder](https://mlsanity.com/paper/2409.09716/) (2024)
- [Autoassociative Learning of Structural Representations for Modeling and Classification in Medical Imaging](https://mlsanity.com/paper/2411.12070/) (2024)
- [HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition](https://mlsanity.com/paper/2607.08249/) (2026)

## Abstract (arXiv)

Representing scenes at the granularity of objects is a prerequisite for scene understanding and decision making. We propose PriSMONet, a novel approach based on Prior Shape knowledge for learning Multi-Object 3D scene decomposition and representations from single images. Our approach learns to decompose images of synthetic scenes with multiple objects on a planar surface into its constituent scene objects and to infer their 3D properties from a single view. A recurrent encoder regresses a latent representation of 3D shape, pose and texture of each object from an input RGB image. By differentiable rendering, we train our model to decompose scenes from RGB-D images in a self-supervised way. The 3D shapes are represented continuously in function-space as signed distance functions which we pre-train from example shapes in a supervised way. These shape priors provide weak supervision signals to better condition the challenging overall learning task. We evaluate the accuracy of our model in inferring 3D scene layout, demonstrate its generative capabilities, assess its generalization to real images, and point out benefits of the learned representation.