# Unsupervised Learning of Depth and Ego-Motion from Cylindrical Panoramic Video with Applications for Virtual Reality

arXiv: https://arxiv.org/abs/2010.07704 · HTML: https://mlsanity.com/paper/2010.07704/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 4 of 4)

- [On the Sins of Image Synthesis Loss for Self-supervised Depth Estimation](https://mlsanity.com/paper/2109.06163/) (2021)
- [Projective Urban Texturing](https://mlsanity.com/paper/2201.10938/) (2022)
- [A high-precision self-supervised monocular visual odometry in foggy weather based on robust cycled generative adversarial networks and multi-task learning aided depth estimation](https://mlsanity.com/paper/2203.04812/) (2022)
- [Distortion-Aware Self-Supervised 360deg Depth Estimation from A Single Equirectangular Projection Image](https://mlsanity.com/paper/2204.01027/) (2022)

## Abstract (arXiv)

We introduce a convolutional neural network model for unsupervised learning of depth and ego-motion from cylindrical panoramic video. Panoramic depth estimation is an important technology for applications such as virtual reality, 3D modeling, and autonomous robotic navigation. In contrast to previous approaches for applying convolutional neural networks to panoramic imagery, we use the cylindrical panoramic projection which allows for the use of the traditional CNN layers such as convolutional filters and max pooling without modification. Our evaluation of synthetic and real data shows that unsupervised learning of depth and ego-motion on cylindrical panoramic images can produce high-quality depth maps and that an increased field-of-view improves ego-motion estimation accuracy. We create two new datasets to evaluate our approach: a synthetic dataset created using the CARLA simulator, and Headcam, a novel dataset of panoramic video collected from a helmet-mounted camera while biking in an urban setting. We also apply our network to the problem of converting monocular panoramas to stereo panoramas.