# Unsupervised Learning of Depth and Ego-Motion from Video

arXiv: https://arxiv.org/abs/1704.07813 · HTML: https://mlsanity.com/paper/1704.07813/

## Key facts

- Cited by 782 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2017.

## Built on by (most cited first, 30 of 782)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [Cross-view Transformers for real-time Map-view Semantic Segmentation](https://mlsanity.com/paper/2205.02833/) (2022)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [Is Pseudo-Lidar needed for Monocular 3D Object detection?](https://mlsanity.com/paper/2108.06417/) (2021)
- [NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior](https://mlsanity.com/paper/2212.07388/) (2022)
- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [Robust Dynamic Radiance Fields](https://mlsanity.com/paper/2301.02239/) (2023)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [Probabilistic and Geometric Depth: Detecting Objects in Perspective](https://mlsanity.com/paper/2107.14160/) (2021)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.13132/) (2022)
- [CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion](https://mlsanity.com/paper/2210.10716/) (2022)
- [PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment](https://mlsanity.com/paper/2306.15667/) (2023)
- [DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing](https://mlsanity.com/paper/2403.17822/) (2024)
- [MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer](https://mlsanity.com/paper/2208.03543/) (2022)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentation](https://mlsanity.com/paper/2104.04691/) (2021)
- [SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction](https://mlsanity.com/paper/2311.12754/) (2023)
- [NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors](https://mlsanity.com/paper/2206.13597/) (2022)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image](https://mlsanity.com/paper/2406.04343/) (2024)
- [Map-free Visual Relocalization: Metric Pose Relative to a Single Image](https://mlsanity.com/paper/2210.05494/) (2022)
- [P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior](https://mlsanity.com/paper/2204.02091/) (2022)
- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)

## Abstract (arXiv)

We present an unsupervised learning framework for the task of monocular depth and camera motion estimation from unstructured video sequences. We achieve this by simultaneously training depth and camera pose estimation networks using the task of view synthesis as the supervisory signal. The networks are thus coupled via the view synthesis objective during training, but can be applied independently at test time. Empirical evaluation on the KITTI dataset demonstrates the effectiveness of our approach: 1) monocular depth performing comparably with supervised methods that use either ground-truth pose or depth for training, and 2) pose estimation performing favorably with established SLAM systems under comparable input settings.