# Enhancing Interpretable Object Abstraction via Clustering-based Slot Initialization

arXiv: https://arxiv.org/abs/2308.11369 · HTML: https://mlsanity.com/paper/2308.11369/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Conditional Object-Centric Learning from Video](https://mlsanity.com/paper/2111.12594/) (2021)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations](https://mlsanity.com/paper/1907.13052/) (2019)
- [Attend, Infer, Repeat: Fast Scene Understanding with Generative Models](https://mlsanity.com/paper/1603.08575/) (2016)
- [Self-supervised Video Object Segmentation by Motion Grouping](https://mlsanity.com/paper/2104.07658/) (2021)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization](https://mlsanity.com/paper/1704.06327/) (2017)
- [Entity Abstraction in Visual Model-Based Reinforcement Learning](https://mlsanity.com/paper/1910.12827/) (2019)
- [Unsupervised Discovery of Object Radiance Fields](https://mlsanity.com/paper/2107.07905/) (2021)
- [Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentation](https://mlsanity.com/paper/2104.01148/) (2021)
- [SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition](https://mlsanity.com/paper/2106.03849/) (2021)
- [DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Images](https://mlsanity.com/paper/2208.07227/) (2022)
- [Toward unsupervised, multi-object discovery in large-scale image collections](https://mlsanity.com/paper/2007.02662/) (2020)
- [Large-Scale Unsupervised Object Discovery](https://mlsanity.com/paper/2106.06650/) (2021)
- [Deep k-Means: Jointly clustering with k-Means and learning representations](https://mlsanity.com/paper/1806.10069/) (2018)
- [Discovering Objects that Can Move](https://mlsanity.com/paper/2203.10159/) (2022)
- [FWD: Real-time Novel View Synthesis with Forward Warping and Depth](https://mlsanity.com/paper/2206.08355/) (2022)
- [Towards causal generative scene models via competition of experts](https://mlsanity.com/paper/2004.12906/) (2020)
- [Learning Object-Centric Representations of Multi-Object Scenes from Multiple Views](https://mlsanity.com/paper/2111.07117/) (2021)
- [SCOUTER: Slot Attention-based Classifier for Explainable Image Recognition](https://mlsanity.com/paper/2009.06138/) (2020)
- [Benchmarking Unsupervised Object Representations for Video Sequences](https://mlsanity.com/paper/2006.07034/) (2020)

## Built on by (most cited first, 3 of 3)

- [SA6D: Self-Adaptive Few-Shot 6D Pose Estimator for Novel and Occluded Objects](https://mlsanity.com/paper/2308.16528/) (2023)
- [Simplified priors for Object-Centric Learning](https://mlsanity.com/paper/2410.00728/) (2024)
- [HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition](https://mlsanity.com/paper/2607.08249/) (2026)

## Abstract (arXiv)

Object-centric representations using slots have shown the advances towards efficient, flexible and interpretable abstraction from low-level perceptual features in a compositional scene. Current approaches randomize the initial state of slots followed by an iterative refinement. As we show in this paper, the random slot initialization significantly affects the accuracy of the final slot prediction. Moreover, current approaches require a predetermined number of slots from prior knowledge of the data, which limits the applicability in the real world. In our work, we initialize the slot representations with clustering algorithms conditioned on the perceptual input features. This requires an additional layer in the architecture to initialize the slots given the identified clusters. We design permutation invariant and permutation equivariant versions of this layer to enable the exchangeable slot representations after clustering. Additionally, we employ mean-shift clustering to automatically identify the number of slots for a given scene. We evaluate our method on object discovery and novel view synthesis tasks with various datasets. The results show that our method outperforms prior works consistently, especially for complex scenes.