# Self-supervised Learning for Single View Depth and Surface Normal Estimation

arXiv: https://arxiv.org/abs/1903.00112 · HTML: https://mlsanity.com/paper/1903.00112/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Mar 2019.

## Built on by (most cited first, 8 of 8)

- [Learning Optical Flow, Depth, and Scene Flow without Real-World Labels](https://mlsanity.com/paper/2203.15089/) (2022)
- [DF-VO: What Should Be Learnt for Visual Odometry?](https://mlsanity.com/paper/2103.00933/) (2021)
- [ColDE: A Depth Estimation Framework for Colonoscopy Reconstruction](https://mlsanity.com/paper/2111.10371/) (2021)
- [DeepShadow: Neural Shape from Shadow](https://mlsanity.com/paper/2203.15065/) (2022)
- [Self-Supervised Depth Completion for Active Stereo](https://mlsanity.com/paper/2110.03234/) (2021)
- [Revisiting PatchMatch Multi-View Stereo for Urban 3D Reconstruction](https://mlsanity.com/paper/2207.08439/) (2022)
- [CbwLoss: Constrained Bidirectional Weighted Loss for Self-supervised Learning of Depth and Pose](https://mlsanity.com/paper/2212.05845/) (2022)
- [Robust and Real-time Surface Normal Estimation from Stereo Disparities using Affine Transformations](https://mlsanity.com/paper/2504.15121/) (2025)

## Abstract (arXiv)

In this work we present a self-supervised learning framework to simultaneously train two Convolutional Neural Networks (CNNs) to predict depth and surface normals from a single image. In contrast to most existing frameworks which represent outdoor scenes as fronto-parallel planes at piece-wise smooth depth, we propose to predict depth with surface orientation while assuming that natural scenes have piece-wise smooth normals. We show that a simple depth-normal consistency as a soft-constraint on the predictions is sufficient and effective for training both these networks simultaneously. The trained normal network provides state-of-the-art predictions while the depth network, relying on much realistic smooth normal assumption, outperforms the traditional self-supervised depth prediction network by a large margin on the KITTI benchmark. Demo video: https://youtu.be/ZD-ZRsw7hdM