# Object Discovery in Videos as Foreground Motion Clustering

arXiv: https://arxiv.org/abs/1812.02772 · HTML: https://mlsanity.com/paper/1812.02772/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2018.

## Built on by (most cited first, 30 of 32)

- [Self-supervised Video Object Segmentation by Motion Grouping](https://mlsanity.com/paper/2104.07658/) (2021)
- [D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video](https://mlsanity.com/paper/2205.15838/) (2022)
- [Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition](https://mlsanity.com/paper/2302.11566/) (2023)
- [Segmenting Moving Objects via an Object-Centric Layered Representation](https://mlsanity.com/paper/2207.02206/) (2022)
- [Deformable Sprites for Unsupervised Video Decomposition](https://mlsanity.com/paper/2204.07151/) (2022)
- [NeuralDiff: Segmenting 3D objects that move in egocentric videos](https://mlsanity.com/paper/2110.09936/) (2021)
- [Opening up Open-World Tracking](https://mlsanity.com/paper/2104.11221/) (2021)
- [Advances in Deep Concealed Scene Understanding](https://mlsanity.com/paper/2304.11234/) (2023)
- [Motion-aware Contrastive Video Representation Learning via Foreground-background Merging](https://mlsanity.com/paper/2109.15130/) (2021)
- [Discovering Objects that Can Move](https://mlsanity.com/paper/2203.10159/) (2022)
- [A Survey of Camouflaged Object Detection and Beyond](https://mlsanity.com/paper/2408.14562/) (2024)
- [Unsupervised Object Learning via Common Fate](https://mlsanity.com/paper/2110.06562/) (2021)
- [Mean Shift Mask Transformer for Unseen Object Instance Segmentation](https://mlsanity.com/paper/2211.11679/) (2022)
- [Unknown Object Segmentation from Stereo Images](https://mlsanity.com/paper/2103.06796/) (2021)
- [Multi-Object Discovery by Low-Dimensional Object Motion](https://mlsanity.com/paper/2307.08027/) (2023)
- [RICE: Refining Instance Masks in Cluttered Environments with Graph Neural Networks](https://mlsanity.com/paper/2106.15711/) (2021)
- [A Simple and Powerful Global Optimization for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2209.09341/) (2022)
- [Motion-inductive Self-supervised Object Discovery in Videos](https://mlsanity.com/paper/2210.00221/) (2022)
- [The Pursuit of Knowledge: Discovering and Localizing Novel Categories using Dual Memory](https://mlsanity.com/paper/2105.01652/) (2021)
- [MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Field](https://mlsanity.com/paper/2303.05703/) (2023)
- [Appearance-Based Refinement for Object-Centric Motion Segmentation](https://mlsanity.com/paper/2312.11463/) (2023)
- [Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation](https://mlsanity.com/paper/2311.17893/) (2023)
- [Event-Free Moving Object Segmentation from Moving Ego Vehicle](https://mlsanity.com/paper/2305.00126/) (2023)
- [DEVIAS: Learning Disentangled Video Representations of Action and Scene](https://mlsanity.com/paper/2312.00826/) (2023)
- [Online Unsupervised Video Object Segmentation via Contrastive Motion Clustering](https://mlsanity.com/paper/2306.12048/) (2023)
- [On Moving Object Segmentation from Monocular Video with Transformers](https://mlsanity.com/paper/2411.19141/) (2024)
- [A topological solution to object segmentation and tracking](https://mlsanity.com/paper/2107.02036/) (2021)
- [RiWNet: A moving object instance segmentation Network being Robust in adverse Weather conditions](https://mlsanity.com/paper/2109.01820/) (2021)
- [MOD-UV: Learning Mobile Object Detectors from Unlabeled Videos](https://mlsanity.com/paper/2405.14841/) (2024)
- [Maximal Cliques on Multi-Frame Proposal Graph for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2301.12352/) (2023)

## Abstract (arXiv)

We consider the problem of providing dense segmentation masks for object discovery in videos. We formulate the object discovery problem as foreground motion clustering, where the goal is to cluster foreground pixels in videos into different objects. We introduce a novel pixel-trajectory recurrent neural network that learns feature embeddings of foreground pixel trajectories linked across time. By clustering the pixel trajectories using the learned feature embeddings, our method establishes correspondences between foreground object masks across video frames. To demonstrate the effectiveness of our framework for object discovery, we conduct experiments on commonly used datasets for motion segmentation, where we achieve state-of-the-art performance.