# The Edge of Depth: Explicit Constraints between Segmentation and Depth

arXiv: https://arxiv.org/abs/2004.00171 · HTML: https://mlsanity.com/paper/2004.00171/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2020.

## Built on by (most cited first, 30 of 72)

- [BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation](https://mlsanity.com/paper/2204.00987/) (2022)
- [P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior](https://mlsanity.com/paper/2204.02091/) (2022)
- [Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging](https://mlsanity.com/paper/2105.14021/) (2021)
- [Self-Supervised Monocular Depth Estimation with Internal Feature Fusion](https://mlsanity.com/paper/2110.09482/) (2021)
- [Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2108.08829/) (2021)
- [Radar-Camera Pixel Depth Association for Depth Completion](https://mlsanity.com/paper/2106.02778/) (2021)
- [Excavating the Potential Capacity of Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2109.12484/) (2021)
- [Depth Completion with Twin Surface Extrapolation at Occlusion Boundaries](https://mlsanity.com/paper/2104.02253/) (2021)
- [RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2207.11984/) (2022)
- [BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimation](https://mlsanity.com/paper/2209.02952/) (2022)
- [Tame a Wild Camera: In-the-Wild Monocular Camera Calibration](https://mlsanity.com/paper/2306.10988/) (2023)
- [PlaneDepth: Self-supervised Depth Estimation via Orthogonal Planes](https://mlsanity.com/paper/2210.01612/) (2022)
- [PLADE-Net: Towards Pixel-Level Accuracy for Self-Supervised Single-View Depth Estimation with Neural Positional Encoding and Distilled Matting Loss](https://mlsanity.com/paper/2103.07362/) (2021)
- [SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras](https://mlsanity.com/paper/2104.04420/) (2021)
- [DevNet: Self-supervised Monocular Depth Learning via Density Volume Construction](https://mlsanity.com/paper/2209.06351/) (2022)
- [SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimation](https://mlsanity.com/paper/2309.00526/) (2023)
- [Self-Supervised Monocular Depth Estimation: Solving the Edge-Fattening Problem](https://mlsanity.com/paper/2210.00411/) (2022)
- [Learning Depth via Leveraging Semantics: Self-supervised Monocular Depth Estimation with Both Implicit and Explicit Semantic Guidance](https://mlsanity.com/paper/2102.06685/) (2021)
- [Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking](https://mlsanity.com/paper/2210.16674/) (2022)
- [EVEN: An Event-Based Framework for Monocular Depth Estimation at Adverse Night Conditions](https://mlsanity.com/paper/2302.03860/) (2023)
- [Self-distilled Feature Aggregation for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2209.07088/) (2022)
- [Detecting Adversarial Perturbations in Multi-Task Perception](https://mlsanity.com/paper/2203.01177/) (2022)
- [Towards Interpretable Deep Networks for Monocular Depth Estimation](https://mlsanity.com/paper/2108.05312/) (2021)
- [X-Distill: Improving Self-Supervised Monocular Depth via Cross-Task Distillation](https://mlsanity.com/paper/2110.12516/) (2021)
- [Temporally Consistent Online Depth Estimation Using Point-Based Fusion](https://mlsanity.com/paper/2304.07435/) (2023)
- [Delving into Crispness: Guided Label Refinement for Crisp Edge Detection](https://mlsanity.com/paper/2306.15172/) (2023)
- [THE Benchmark: Transferable Representation Learning for Monocular Height Estimation](https://mlsanity.com/paper/2112.14985/) (2021)
- [Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2212.05315/) (2022)
- [Two-in-One Depth: Bridging the Gap Between Monocular and Binocular Self-supervised Depth Estimation](https://mlsanity.com/paper/2309.00933/) (2023)
- [Joint Depth Prediction and Semantic Segmentation with Multi-View SAM](https://mlsanity.com/paper/2311.00134/) (2023)

## Abstract (arXiv)

In this work we study the mutual benefits of two common computer vision tasks, self-supervised depth estimation and semantic segmentation from images. For example, to help unsupervised monocular depth estimation, constraints from semantic segmentation has been explored implicitly such as sharing and transforming features. In contrast, we propose to explicitly measure the border consistency between segmentation and depth and minimize it in a greedy manner by iteratively supervising the network towards a locally optimal solution. Partially this is motivated by our observation that semantic segmentation even trained with limited ground truth (200 images of KITTI) can offer more accurate border than that of any (monocular or stereo) image-based depth estimation. Through extensive experiments, our proposed approach advances the state of the art on unsupervised monocular depth estimation in the KITTI.