# Learning Independent Object Motion from Unlabelled Stereoscopic Videos

arXiv: https://arxiv.org/abs/1901.01971 · HTML: https://mlsanity.com/paper/1901.01971/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2019.

## Built on by (most cited first, 17 of 17)

- [Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency](https://mlsanity.com/paper/2102.02629/) (2021)
- [Segment Any Motion in Videos](https://mlsanity.com/paper/2503.22268/) (2025)
- [Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes](https://mlsanity.com/paper/2304.08993/) (2023)
- [Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation](https://mlsanity.com/paper/2110.06853/) (2021)
- [RoDUS: Robust Decomposition of Static and Dynamic Elements in Urban Scenes](https://mlsanity.com/paper/2403.09419/) (2024)
- [Self-Supervised Monocular Scene Decomposition and Depth Estimation](https://mlsanity.com/paper/2110.11275/) (2021)
- [Motion Segmentation from a Moving Monocular Camera](https://mlsanity.com/paper/2309.13772/) (2023)
- [Instance-Level Moving Object Segmentation from a Single Image with Events](https://mlsanity.com/paper/2502.12975/) (2025)
- [OmniSyn: Synthesizing 360 Videos with Wide-baseline Panoramas](https://mlsanity.com/paper/2202.08752/) (2022)
- [EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigidity](https://mlsanity.com/paper/2309.01296/) (2023)
- [Zero-Shot Monocular Motion Segmentation in the Wild by Combining Deep Learning with Geometric Motion Model Fusion](https://mlsanity.com/paper/2405.01723/) (2024)
- [SM3D: Simultaneous Monocular Mapping and 3D Detection](https://mlsanity.com/paper/2111.12643/) (2021)
- [Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance](https://mlsanity.com/paper/2209.04945/) (2022)
- [MoDA: Leveraging Motion Priors from Videos for Advancing Unsupervised Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2309.11711/) (2023)
- [USegScene: Unsupervised Learning of Depth, Optical Flow and Ego-Motion with Semantic Guidance and Coupled Networks](https://mlsanity.com/paper/2207.07469/) (2022)
- [DO3D: Self-supervised Learning of Decomposed Object-aware 3D Motion and Depth from Monocular Videos](https://mlsanity.com/paper/2403.05895/) (2024)
- [SUMO: Segment and Track Any Motion with Nonlinear State Space Models](https://mlsanity.com/paper/2606.29861/) (2026)

## Abstract (arXiv)

We present a system for learning motion of independently moving objects from stereo videos. The only human annotation used in our system are 2D object bounding boxes which introduce the notion of objects to our system. Unlike prior learning based work which has focused on predicting dense pixel-wise optical flow field and/or a depth map for each image, we propose to predict object instance specific 3D scene flow maps and instance masks from which we are able to derive the motion direction and speed for each object instance. Our network takes the 3D geometry of the problem into account which allows it to correlate the input images. We present experiments evaluating the accuracy of our 3D flow vectors, as well as depth maps and projected 2D optical flow where our jointly learned system outperforms earlier approaches trained for each task independently.