# Distortion-Aware Self-Supervised 360deg Depth Estimation from A Single Equirectangular Projection Image

arXiv: https://arxiv.org/abs/2204.01027 · HTML: https://mlsanity.com/paper/2204.01027/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 3 Apr 2022.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing](https://mlsanity.com/paper/1805.04409/) (2018)
- [Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera](https://mlsanity.com/paper/2004.01294/) (2020)
- [Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation](https://mlsanity.com/paper/1906.03525/) (2019)
- [Learning the Depths of Moving People by Watching Frozen People](https://mlsanity.com/paper/1904.11111/) (2019)
- [OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas](https://mlsanity.com/paper/1807.09620/) (2018)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)
- [Orientation-aware Semantic Segmentation on Icosahedron Spheres](https://mlsanity.com/paper/1907.12849/) (2019)
- [Self-Supervised Learning of Depth and Camera Motion from 360deg Videos](https://mlsanity.com/paper/1811.05304/) (2018)
- [Object Detection in Equirectangular Panorama](https://mlsanity.com/paper/1805.08009/) (2018)
- [Super-resolution of Omnidirectional Images Using Adversarial Learning](https://mlsanity.com/paper/1908.04297/) (2019)
- [Unsupervised Learning of Depth and Ego-Motion from Cylindrical Panoramic Video with Applications for Virtual Reality](https://mlsanity.com/paper/2010.07704/) (2020)
- [Building Movie Map -- A Tool for Exploring Areas in a City -- and its Evaluation](https://mlsanity.com/paper/2011.08525/) (2020)

## Abstract (arXiv)

360deg images are widely available over the last few years. This paper proposes a new technique for single 360deg image depth prediction under open environments. Depth prediction from a 360deg single image is not easy for two reasons. One is the limitation of supervision datasets - the currently available dataset is limited to indoor scenes. The other is the problems caused by Equirectangular Projection Format (ERP), commonly used for 360deg images, that are coordinate and distortion. There is only one method existing that uses cube map projection to produce six perspective images and apply self-supervised learning using motion pictures for perspective depth prediction to deal with these problems. Different from the existing method, we directly use the ERP format. We propose a framework of direct use of ERP with coordinate conversion of correspondences and distortion-aware upsampling module to deal with the ERP related problems and extend a self-supervised learning method for open environments. For the experiments, we firstly built a dataset for the evaluation, and quantitatively evaluate the depth prediction in outdoor scenes. We show that it outperforms the state-of-the-art technique