# Self-supervised Object Motion and Depth Estimation from Video

arXiv: https://arxiv.org/abs/1912.04250 · HTML: https://mlsanity.com/paper/1912.04250/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2019.

## Built on by (most cited first, 17 of 17)

- [Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2108.12545/) (2021)
- [Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation](https://mlsanity.com/paper/2110.06853/) (2021)
- [Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTV](https://mlsanity.com/paper/2307.10713/) (2023)
- [EGA-Depth: Efficient Guided Attention for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2304.03369/) (2023)
- [X-Distill: Improving Self-Supervised Monocular Depth via Cross-Task Distillation](https://mlsanity.com/paper/2110.12516/) (2021)
- [The Second Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2304.07051/) (2023)
- [The Third Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2404.16831/) (2024)
- [Cross-spectral Gated-RGB Stereo Depth Estimation](https://mlsanity.com/paper/2405.12759/) (2024)
- [The Fourth Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2504.17787/) (2025)
- [Panoptic Segmentation Forecasting](https://mlsanity.com/paper/2104.03962/) (2021)
- [Gated Stereo: Joint Depth Estimation from Gated and Wide-Baseline Active Stereo Cues](https://mlsanity.com/paper/2305.12955/) (2023)
- [Kick Back & Relax++: Scaling Beyond Ground-Truth Depth with SlowTV & CribsTV](https://mlsanity.com/paper/2403.01569/) (2024)
- [Gated2Gated: Self-Supervised Depth Estimation from Gated Images](https://mlsanity.com/paper/2112.02416/) (2021)
- [Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Video](https://mlsanity.com/paper/2105.14520/) (2021)
- [Joint Forecasting of Panoptic Segmentations with Difference Attention](https://mlsanity.com/paper/2204.07157/) (2022)
- [Self-Supervised Depth Estimation with Isometric-Self-Sample-Based Learning](https://mlsanity.com/paper/2205.10006/) (2022)
- [Monocular Depth Estimation from a Single Image: Progress and Opportunities](https://mlsanity.com/paper/2609.01172/) (2026)

## Abstract (arXiv)

We present a self-supervised learning framework to estimate the individual object motion and monocular depth from video. We model the object motion as a 6 degree-of-freedom rigid-body transformation. The instance segmentation mask is leveraged to introduce the information of object. Compared with methods which predict dense optical flow map to model the motion, our approach significantly reduces the number of values to be estimated. Our system eliminates the scale ambiguity of motion prediction through imposing a novel geometric constraint loss term. Experiments on KITTI driving dataset demonstrate our system is capable to capture the object motion without external annotation. Our system outperforms previous self-supervised approaches in terms of 3D scene flow prediction, and contribute to the disparity prediction in dynamic area.