# Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency

arXiv: https://arxiv.org/abs/1711.03665 · HTML: https://mlsanity.com/paper/1711.03665/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2017.

## Built on by (most cited first, 30 of 40)

- [CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion](https://mlsanity.com/paper/2210.10716/) (2022)
- [MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer](https://mlsanity.com/paper/2208.03543/) (2022)
- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2112.13047/) (2021)
- [Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark](https://mlsanity.com/paper/2108.03830/) (2021)
- [Radar-Camera Pixel Depth Association for Depth Completion](https://mlsanity.com/paper/2106.02778/) (2021)
- [Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry](https://mlsanity.com/paper/2112.08177/) (2021)
- [DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learning](https://mlsanity.com/paper/2206.08927/) (2022)
- [Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking](https://mlsanity.com/paper/2210.16674/) (2022)
- [Vision Transformer Adapters for Generalizable Multitask Learning](https://mlsanity.com/paper/2308.12372/) (2023)
- [Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR](https://mlsanity.com/paper/2109.09628/) (2021)
- [MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Model](https://mlsanity.com/paper/2311.07198/) (2023)
- [GCNDepth: Self-supervised Monocular Depth Estimation based on Graph Convolutional Network](https://mlsanity.com/paper/2112.06782/) (2021)
- [SfM-TTR: Using Structure from Motion for Test-Time Refinement of Single-View Depth Networks](https://mlsanity.com/paper/2211.13551/) (2022)
- [SwinDepth: Unsupervised Depth Estimation using Monocular Sequences via Swin Transformer and Densely Cascaded Network](https://mlsanity.com/paper/2301.06715/) (2023)
- [Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement](https://mlsanity.com/paper/2302.09789/) (2023)
- [Moving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenes](https://mlsanity.com/paper/2105.02195/) (2021)
- [Estimating Image Depth in the Comics Domain](https://mlsanity.com/paper/2110.03575/) (2021)
- [GeoRefine: Self-Supervised Online Depth Refinement for Accurate Dense Mapping](https://mlsanity.com/paper/2205.01656/) (2022)
- [Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision](https://mlsanity.com/paper/2302.09922/) (2023)
- [SPIdepth: Strengthened Pose Information for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2404.12501/) (2024)
- [Self-supervised Depth Estimation Leveraging Global Perception and Geometric Smoothness Using On-board Videos](https://mlsanity.com/paper/2106.03505/) (2021)
- [DnD: Dense Depth Estimation in Crowded Dynamic Indoor Scenes](https://mlsanity.com/paper/2108.05615/) (2021)
- [Instance-aware multi-object self-supervision for monocular depth prediction](https://mlsanity.com/paper/2203.00809/) (2022)
- [Road-aware Monocular Structure from Motion and Homography Estimation](https://mlsanity.com/paper/2112.08635/) (2021)
- [Forecasting of depth and ego-motion with transformers and self-supervision](https://mlsanity.com/paper/2206.07435/) (2022)
- [ADAADepth: Adapting Data Augmentation and Attention for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2103.00853/) (2021)
- [Efficient Stereo Depth Estimation for Pseudo LiDAR: A Self-Supervised Approach Based on Multi-Input ResNet Encoder](https://mlsanity.com/paper/2205.08089/) (2022)
- [Self-Guided Instance-Aware Network for Depth Completion and Enhancement](https://mlsanity.com/paper/2105.12186/) (2021)
- [Geometric Constraints in Deep Learning Frameworks: A Survey](https://mlsanity.com/paper/2403.12431/) (2024)

## Abstract (arXiv)

Learning to reconstruct depths in a single image by watching unlabeled videos via deep convolutional network (DCN) is attracting significant attention in recent years. In this paper, we introduce a surface normal representation for unsupervised depth estimation framework. Our estimated depths are constrained to be compatible with predicted normals, yielding more robust geometry results. Specifically, we formulate an edge-aware depth-normal consistency term, and solve it by constructing a depth-to-normal layer and a normal-to-depth layer inside of the DCN. The depth-to-normal layer takes estimated depths as input, and computes normal directions using cross production based on neighboring pixels. Then given the estimated normals, the normal-to-depth layer outputs a regularized depth map through local planar smoothness. Both layers are computed with awareness of edges inside the image to help address the issue of depth/normal discontinuity and preserve sharp edges. Finally, to train the network, we apply the photometric error and gradient smoothness for both depth and normal predictions. We conducted experiments on both outdoor (KITTI) and indoor (NYUv2) datasets, and show that our algorithm vastly outperforms state of the art, which demonstrates the benefits from our approach.