# 3D Rigid Motion Segmentation with Mixed and Unknown Number of Models

arXiv: https://arxiv.org/abs/1908.06087 · HTML: https://mlsanity.com/paper/1908.06087/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2019.

## Built on by (most cited first, 11 of 11)

- [Learning to Segment Rigid Motions from Two Frames](https://mlsanity.com/paper/2101.03694/) (2021)
- [MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization](https://mlsanity.com/paper/2101.06605/) (2021)
- [Morig: Motion-aware rigging of character meshes from point clouds](https://mlsanity.com/paper/2210.09463/) (2022)
- [Quantum Motion Segmentation](https://mlsanity.com/paper/2203.13185/) (2022)
- [Multi-body SE(3) Equivariance for Unsupervised Rigid Segmentation and Motion Estimation](https://mlsanity.com/paper/2306.05584/) (2023)
- [On Moving Object Segmentation from Monocular Video with Transformers](https://mlsanity.com/paper/2411.19141/) (2024)
- [RiWNet: A moving object instance segmentation Network being Robust in adverse Weather conditions](https://mlsanity.com/paper/2109.01820/) (2021)
- [Latent Semantic Consensus For Deterministic Geometric Model Fitting](https://mlsanity.com/paper/2403.06444/) (2024)
- [MultiLink: Multi-class Structure Recovery via Agglomerative Clustering and Model Selection](https://mlsanity.com/paper/2505.10874/) (2025)
- [UFD-PRiME: Unsupervised Joint Learning of Optical Flow and Stereo Depth through Pixel-Level Rigid Motion Estimation](https://mlsanity.com/paper/2310.04712/) (2023)
- [APES: Articulated Part Extraction from Sprite Sheets](https://mlsanity.com/paper/2206.02015/) (2022)

## Abstract (arXiv)

Many real-world video sequences cannot be conveniently categorized as general or degenerate; in such cases, imposing a false dichotomy in using the fundamental matrix or homography model for motion segmentation on video sequences would lead to difficulty. Even when we are confronted with a general scene-motion, the fundamental matrix approach as a model for motion segmentation still suffers from several defects, which we discuss in this paper. The full potential of the fundamental matrix approach could only be realized if we judiciously harness information from the simpler homography model. From these considerations, we propose a multi-model spectral clustering framework that synergistically combines multiple models (homography and fundamental matrix) together. We show that the performance can be substantially improved in this way. For general motion segmentation tasks, the number of independently moving objects is often unknown a priori and needs to be estimated from the observations. This is referred to as model selection and it is essentially still an open research problem. In this work, we propose a set of model selection criteria balancing data fidelity and model complexity. We perform extensive testing on existing motion segmentation datasets with both segmentation and model selection tasks, achieving state-of-the-art performance on all of them; we also put forth a more realistic and challenging dataset adapted from the KITTI benchmark, containing real-world effects such as strong perspectives and strong forward translations not seen in the traditional datasets.