# CRF360D: Monocular 360 Depth Estimation via Spherical Fully-Connected CRFs

arXiv: https://arxiv.org/abs/2405.11564 · HTML: https://mlsanity.com/paper/2405.11564/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 33 references that are in the map.
- First posted on arXiv 19 May 2024.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Conditional Positional Encodings for Vision Transformers](https://mlsanity.com/paper/2102.10882/) (2021)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimation](https://mlsanity.com/paper/2203.01502/) (2022)
- [HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features](https://mlsanity.com/paper/2011.11498/) (2020)
- [OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas](https://mlsanity.com/paper/1807.09620/) (2018)
- [PanoFormer: Panorama Transformer for Indoor 360 Depth Estimation](https://mlsanity.com/paper/2203.09283/) (2022)
- [UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimation](https://mlsanity.com/paper/2102.03550/) (2021)
- [OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion](https://mlsanity.com/paper/2203.00838/) (2022)
- [Towards Real-Time Monocular Depth Estimation for Robotics: A Survey](https://mlsanity.com/paper/2111.08600/) (2021)
- [EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimation](https://mlsanity.com/paper/2304.07803/) (2023)
- [Self-Supervised Learning of Depth and Camera Motion from 360deg Videos](https://mlsanity.com/paper/1811.05304/) (2018)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [HRDFuse: Monocular 360degDepth Estimation by Collaboratively Learning Holistic-with-Regional Depth Distributions](https://mlsanity.com/paper/2303.11616/) (2023)
- [SphereSR: 360deg Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation](https://mlsanity.com/paper/2112.06536/) (2021)
- [MGNet: Monocular Geometric Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2206.13199/) (2022)
- [PanoSwin: a Pano-style Swin Transformer for Panorama Understanding](https://mlsanity.com/paper/2308.14726/) (2023)
- [S2Net: Accurate Panorama Depth Estimation on Spherical Surface](https://mlsanity.com/paper/2301.05845/) (2023)

## Built on by (most cited first, 1 of 1)

- [VGGT-360: Geometry-Consistent Zero-Shot Panoramic Depth Estimation](https://mlsanity.com/paper/2603.18943/) (2026)

## Abstract (arXiv)

Monocular 360 depth estimation is challenging due to the inherent distortion of the equirectangular projection (ERP). This distortion causes a problem: spherical adjacent points are separated after being projected to the ERP plane, particularly in the polar regions. To tackle this problem, recent methods calculate the spherical neighbors in the tangent domain. However, as the tangent patch and sphere only have one common point, these methods construct neighboring spherical relationships around the common point. In this paper, we propose spherical fully-connected CRFs (SF-CRFs). We begin by evenly partitioning an ERP image with regular windows, where windows at the equator involve broader spherical neighbors than those at the poles. To improve the spherical relationships, our SF-CRFs enjoy two key components. Firstly, to involve sufficient spherical neighbors, we propose a Spherical Window Transform (SWT) module. This module aims to replicate the equator window's spherical relationships to all other windows, leveraging the rotational invariance of the sphere. Remarkably, the transformation process is highly efficient, completing the transformation of all windows in a 512X1024 ERP with 0.038 seconds on CPU. Secondly, we propose a Planar-Spherical Interaction (PSI) module to facilitate the relationships between regular and transformed windows, which not only preserves the local details but also captures global structures. By building a decoder based on the SF-CRFs blocks, we propose CRF360D, a novel 360 depth estimation framework that achieves state-of-the-art performance across diverse datasets. Our CRF360D is compatible with different perspective image-trained backbones (e.g., EfficientNet), serving as the encoder.