# Exploiting Spatial Invariance for Scalable Unsupervised Object Tracking

arXiv: https://arxiv.org/abs/1911.09033 · HTML: https://mlsanity.com/paper/1911.09033/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2019.

## Built on by (most cited first, 30 of 40)

- [Conditional Object-Centric Learning from Video](https://mlsanity.com/paper/2111.12594/) (2021)
- [Bridging the Gap to Real-World Object-Centric Learning](https://mlsanity.com/paper/2209.14860/) (2022)
- [Illiterate DALL-E Learns to Compose](https://mlsanity.com/paper/2110.11405/) (2021)
- [Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos](https://mlsanity.com/paper/2205.14065/) (2022)
- [Generalization and Robustness Implications in Object-Centric Learning](https://mlsanity.com/paper/2107.00637/) (2021)
- [SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decomposition](https://mlsanity.com/paper/2106.03849/) (2021)
- [Unsupervised Part Discovery from Contrastive Reconstruction](https://mlsanity.com/paper/2111.06349/) (2021)
- [Opening up Open-World Tracking](https://mlsanity.com/paper/2104.11221/) (2021)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion](https://mlsanity.com/paper/2205.07844/) (2022)
- [Neural Systematic Binder](https://mlsanity.com/paper/2211.01177/) (2022)
- [MarioNette: Self-Supervised Sprite Learning](https://mlsanity.com/paper/2104.14553/) (2021)
- [Unsupervised Object Learning via Common Fate](https://mlsanity.com/paper/2110.06562/) (2021)
- [Compositional Scene Representation Learning via Reconstruction: A Survey](https://mlsanity.com/paper/2202.07135/) (2022)
- [Generative Video Transformer: Can Objects be the Words?](https://mlsanity.com/paper/2107.09240/) (2021)
- [Towards Unsupervised Object Detection From LiDAR Point Clouds](https://mlsanity.com/paper/2311.02007/) (2023)
- [Structured World Belief for Reinforcement Learning in POMDP](https://mlsanity.com/paper/2107.08577/) (2021)
- [Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos](https://mlsanity.com/paper/2308.09951/) (2023)
- [Boosting Object Representation Learning via Motion and Object Continuity](https://mlsanity.com/paper/2211.09771/) (2022)
- [Slot State Space Models](https://mlsanity.com/paper/2406.12272/) (2024)
- [Object-Centric Multiple Object Tracking](https://mlsanity.com/paper/2309.00233/) (2023)
- [Motion-inductive Self-supervised Object Discovery in Videos](https://mlsanity.com/paper/2210.00221/) (2022)
- [Inductive Biases for Object-Centric Representations in the Presence of Complex Textures](https://mlsanity.com/paper/2204.08479/) (2022)
- [Unsupervised Learning of Compositional Scene Representations from Multiple Unspecified Viewpoints](https://mlsanity.com/paper/2112.03568/) (2021)
- [Four Principles for Physically Interpretable World Models](https://mlsanity.com/paper/2503.02143/) (2025)
- [Neural Language of Thought Models](https://mlsanity.com/paper/2402.01203/) (2024)
- [GATSBI: Generative Agent-centric Spatio-temporal Object Interaction](https://mlsanity.com/paper/2104.04275/) (2021)
- [Rethinking Image-to-Video Adaptation: An Object-centric Perspective](https://mlsanity.com/paper/2407.06871/) (2024)
- [Unsupervised Video Prediction from a Single Frame by Estimating 3D Dynamic Scene Structure](https://mlsanity.com/paper/2106.09051/) (2021)
- [Unsupervised Object-Centric Learning from Multiple Unspecified Viewpoints](https://mlsanity.com/paper/2401.01922/) (2024)

## Abstract (arXiv)

The ability to detect and track objects in the visual world is a crucial skill for any intelligent agent, as it is a necessary precursor to any object-level reasoning process. Moreover, it is important that agents learn to track objects without supervision (i.e. without access to annotated training videos) since this will allow agents to begin operating in new environments with minimal human assistance. The task of learning to discover and track objects in videos, which we call textitunsupervised object tracking, has grown in prominence in recent years; however, most architectures that address it still struggle to deal with large scenes containing many objects. In the current work, we propose an architecture that scales well to the large-scene, many-object setting by employing spatially invariant computations (convolutions and spatial attention) and representations (a spatially local object specification scheme). In a series of experiments, we demonstrate a number of attractive features of our architecture; most notably, that it outperforms competing methods at tracking objects in cluttered scenes with many objects, and that it can generalize well to videos that are larger and/or contain more objects than videos encountered during training.