# HRDFuse: Monocular 360degDepth Estimation by Collaboratively Learning Holistic-with-Regional Depth Distributions

arXiv: https://arxiv.org/abs/2303.11616 · HTML: https://mlsanity.com/paper/2303.11616/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 55 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [CMT: Convolutional Neural Networks Meet Vision Transformers](https://mlsanity.com/paper/2107.06263/) (2021)
- [Mobile-Former: Bridging MobileNet and Transformer](https://mlsanity.com/paper/2108.05895/) (2021)
- [All in One: Exploring Unified Video-Language Pre-training](https://mlsanity.com/paper/2203.07303/) (2022)
- [Conformer: Local Features Coupling Global Representations for Visual Recognition](https://mlsanity.com/paper/2105.03889/) (2021)
- [OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas](https://mlsanity.com/paper/1807.09620/) (2018)
- [PanoFormer: Panorama Transformer for Indoor 360 Depth Estimation](https://mlsanity.com/paper/2203.09283/) (2022)
- [UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimation](https://mlsanity.com/paper/2102.03550/) (2021)
- [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019)
- [Tangent Images for Mitigating Spherical Distortion](https://mlsanity.com/paper/1912.09390/) (2019)
- [Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation](https://mlsanity.com/paper/2203.01452/) (2022)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videos](https://mlsanity.com/paper/1806.01320/) (2018)
- [OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion](https://mlsanity.com/paper/2203.00838/) (2022)
- [360MonoDepth: High-Resolution 360deg Monocular Depth Estimation](https://mlsanity.com/paper/2111.15669/) (2021)
- [Capturing Omni-Range Context for Omnidirectional Segmentation](https://mlsanity.com/paper/2103.05687/) (2021)

## Built on by (most cited first, 30 of 37)

- [Depth Any Camera: Zero-Shot Metric Depth Estimation from Any Camera](https://mlsanity.com/paper/2501.02464/) (2025)
- [Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation](https://mlsanity.com/paper/2406.12849/) (2024)
- [Elite360D: Towards Efficient 360 Depth Estimation via Semantic- and Distance-Aware Bi-Projection Fusion](https://mlsanity.com/paper/2403.16376/) (2024)
- [DA2: Depth Anything in Any Direction](https://mlsanity.com/paper/2509.26618/) (2025)
- [Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimation](https://mlsanity.com/paper/2512.16913/) (2025)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas](https://mlsanity.com/paper/2306.01531/) (2023)
- [PanDA: Towards Panoramic Depth Anything with Unlabeled Panoramas and Mobius Spatial Augmentation](https://mlsanity.com/paper/2406.13378/) (2024)
- [SphereUFormer: A U-Shaped Transformer for Spherical 360 Perception](https://mlsanity.com/paper/2412.06968/) (2024)
- [One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Image](https://mlsanity.com/paper/2602.19766/) (2026)
- [Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360 Image Outpainting](https://mlsanity.com/paper/2401.10564/) (2024)
- [SGFormer: Spherical Geometry Transformer for 360 Depth Estimation](https://mlsanity.com/paper/2404.14979/) (2024)
- [PanoSplatt3R: Leveraging Perspective Pretraining for Generalized Unposed Wide-Baseline Panorama Reconstruction](https://mlsanity.com/paper/2507.21960/) (2025)
- [UniDAC: Universal Metric Depth Estimation for Any Camera](https://mlsanity.com/paper/2603.27105/) (2026)
- [ResVR: Joint Rescaling and Viewport Rendering of Omnidirectional Images](https://mlsanity.com/paper/2404.16825/) (2024)
- [PanoNormal: Monocular Indoor 360deg Surface Normal Estimation](https://mlsanity.com/paper/2405.18745/) (2024)
- [VGGT-360: Geometry-Consistent Zero-Shot Panoramic Depth Estimation](https://mlsanity.com/paper/2603.18943/) (2026)
- [Estimating Depth of Monocular Panoramic Image with Teacher-Student Model Fusing Equirectangular and Spherical Representations](https://mlsanity.com/paper/2405.16858/) (2024)
- [Revisiting 360 Depth Estimation with PanoGabor: A New Fusion Perspective](https://mlsanity.com/paper/2408.16227/) (2024)
- [RPG360: Robust 360 Depth Estimation with Perspective Foundation Models and Graph Optimization](https://mlsanity.com/paper/2509.23991/) (2025)
- [Wid3R: Wide Field-of-View 3D Reconstruction via Camera Model Conditioning](https://mlsanity.com/paper/2602.05321/) (2026)
- [Elite360M: Efficient 360 Multi-task Learning via Bi-projection Fusion and Cross-task Collaboration](https://mlsanity.com/paper/2408.09336/) (2024)
- [Multi-task Geometric Estimation of Depth and Surface Normal from Monocular 360deg Images](https://mlsanity.com/paper/2411.01749/) (2024)
- [MaGRITTe: Manipulative and Generative 3D Realization from Image, Topview and Text](https://mlsanity.com/paper/2404.00345/) (2024)
- [Depth Anything in 360^circ: Towards Scale Invariance in the Wild](https://mlsanity.com/paper/2512.22819/) (2025)
- [CRF360D: Monocular 360 Depth Estimation via Spherical Fully-Connected CRFs](https://mlsanity.com/paper/2405.11564/) (2024)
- [360-Degree Full-view Image Segmentation by Spherical Convolution compatible with Large-scale Planar Pre-trained Models](https://mlsanity.com/paper/2507.09216/) (2025)
- [DepthAgent: Towards Better Universal Depth Estimation via Sample-wise Expert Selection](https://mlsanity.com/paper/2605.23281/) (2026)
- [Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling](https://mlsanity.com/paper/2606.27745/) (2026)
- [360^circ High-Resolution Depth Estimation via Uncertainty-aware Structural Knowledge Transfer](https://mlsanity.com/paper/2304.07967/) (2023)

## Abstract (arXiv)

Depth estimation from a monocular 360deg image is a burgeoning problem owing to its holistic sensing of a scene. Recently, some methods, eg, OmniFusion, have applied the tangent projection (TP) to represent a 360degimage and predicted depth values via patch-wise regressions, which are merged to get a depth map with equirectangular projection (ERP) format. However, these methods suffer from 1) non-trivial process of merging plenty of patches; 2) capturing less holistic-with-regional contextual information by directly regressing the depth value of each pixel. In this paper, we propose a novel framework, HRDFuse, that subtly combines the potential of convolutional neural networks (CNNs) and transformers by collaboratively learning the textitholistic contextual information from the ERP and the textitregional structural information from the TP. Firstly, we propose a spatial feature alignment (SFA) module that learns feature similarities between the TP and ERP to aggregate the TP features into a complete ERP feature map in a pixel-wise manner. Secondly, we propose a collaborative depth distribution classification (CDDC) module that learns the holistic-with-regional histograms capturing the ERP and TP depth distributions. As such, the final depth values can be predicted as a linear combination of histogram bin centers. Lastly, we adaptively combine the depth predictions from ERP and TP to obtain the final depth map. Extensive experiments show that our method predicts more smooth and accurate depth results while achieving favorably better results than the SOTA methods.