# IronDepth: Iterative Refinement of Single-View Depth using Surface Normal and its Uncertainty

arXiv: https://arxiv.org/abs/2210.03676 · HTML: https://mlsanity.com/paper/2210.03676/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 7 Oct 2022.

## Builds on (most cited first, 30 of 30)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [Enforcing geometric constraints of virtual normal for depth prediction](https://mlsanity.com/paper/1907.12209/) (2019)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [Evaluation of CNN-based Single-Image Depth Estimation Methods](https://mlsanity.com/paper/1805.01328/) (2018)
- [Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation](https://mlsanity.com/paper/2109.09881/) (2021)
- [Semi-Supervised Deep Learning for Monocular Depth Map Prediction](https://mlsanity.com/paper/1702.02706/) (2017)
- [Deep Convolutional Neural Fields for Depth Estimation from a Single Image](https://mlsanity.com/paper/1411.6387/) (2014)
- [Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks](https://mlsanity.com/paper/1605.02305/) (2016)
- [Guiding Monocular Depth Estimation Using Depth-Attention Volume](https://mlsanity.com/paper/2004.02760/) (2020)
- [Designing Deep Networks for Surface Normal Estimation](https://mlsanity.com/paper/1411.4958/) (2014)
- [SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation](https://mlsanity.com/paper/1905.08598/) (2019)
- [One Shot 3D Photography](https://mlsanity.com/paper/2008.12298/) (2020)
- [Adaptive Surface Normal Constraint for Depth Estimation](https://mlsanity.com/paper/2103.15483/) (2021)
- [Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spheres](https://mlsanity.com/paper/1904.05404/) (2019)
- [GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation](https://mlsanity.com/paper/2012.06980/) (2020)
- [Marr Revisited: 2D-3D Alignment via Surface Normal Prediction](https://mlsanity.com/paper/1604.01347/) (2016)
- [Surface Normal Estimation of Tilted Images via Spatial Rectifier](https://mlsanity.com/paper/2007.09264/) (2020)
- [FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image](https://mlsanity.com/paper/1903.12305/) (2019)

## Built on by (most cited first, 30 of 30)

- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models](https://mlsanity.com/paper/2303.11989/) (2023)
- [Rethinking Inductive Biases for Surface Normal Estimation](https://mlsanity.com/paper/2403.00712/) (2024)
- [iDisc: Internal Discretization for Monocular Depth Estimation](https://mlsanity.com/paper/2304.06334/) (2023)
- [DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image](https://mlsanity.com/paper/2311.18610/) (2023)
- [Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting](https://mlsanity.com/paper/2404.19758/) (2024)
- [OGNI-DC: Robust Depth Completion with Optimization-Guided Neural Iterations](https://mlsanity.com/paper/2406.11711/) (2024)
- [UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM](https://mlsanity.com/paper/2306.11048/) (2023)
- [Learning a Depth Covariance Function](https://mlsanity.com/paper/2303.12157/) (2023)
- [Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation](https://mlsanity.com/paper/2306.15662/) (2023)
- [Single Image Depth Prediction Made Better: A Multivariate Gaussian Take](https://mlsanity.com/paper/2303.18164/) (2023)
- [Learning to Visually Localize Sound Sources from Mixtures without Prior Source Knowledge](https://mlsanity.com/paper/2403.17420/) (2024)
- [Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos](https://mlsanity.com/paper/2403.17915/) (2024)
- [GenRC: Generative 3D Room Completion from Sparse Image Collections](https://mlsanity.com/paper/2407.12939/) (2024)
- [Meta-Optimization for Higher Model Generalizability in Single-Image Depth Prediction](https://mlsanity.com/paper/2305.07269/) (2023)
- [LiftFormer: Lifting and Frame Theory Based Monocular Depth Estimation Using Depth and Edge Oriented Subspace Representation](https://mlsanity.com/paper/2604.06576/) (2026)
- [U2RLE: Uncertainty-Guided 2-Stage Room Layout Estimation](https://mlsanity.com/paper/2304.08580/) (2023)
- [Prim2Room: Layout-Controllable Room Mesh Generation from Primitives](https://mlsanity.com/paper/2409.05380/) (2024)
- [InSpaceType: Reconsider Space Type in Indoor Monocular Depth Estimation](https://mlsanity.com/paper/2309.13516/) (2023)
- [MultiDepth: Multi-Sample Priors for Refining Monocular Metric Depth Estimations in Indoor Scenes](https://mlsanity.com/paper/2411.01048/) (2024)
- [HiGS: Hierarchical Generative Scene Framework for Multi-Step Associative Semantic Spatial Composition](https://mlsanity.com/paper/2510.27148/) (2025)
- [VioLA: Aligning Videos to 2D LiDAR Scans](https://mlsanity.com/paper/2311.04783/) (2023)
- [MEDeA: Multi-view Efficient Depth Adjustment](https://mlsanity.com/paper/2406.12048/) (2024)
- [InSpaceType: Dataset and Benchmark for Reconsidering Cross-Space Type Performance in Indoor Monocular Depth](https://mlsanity.com/paper/2408.13708/) (2024)
- [A Practical Approach to Underwater Depth and Surface Normals Estimation](https://mlsanity.com/paper/2410.02072/) (2024)
- [M3Depth: Wavelet-Enhanced Depth Estimation on Mars via Mutual Boosting of Dual-Modal Data](https://mlsanity.com/paper/2505.14159/) (2025)
- [Sparse-View Surface Reconstruction using Gaussian Splatting through High-Confidence Depth Propagation with Normal Priors](https://mlsanity.com/paper/2607.03765/) (2026)
- [Interpretable Human-Label-Free Deep Learning for Real-Bogus Classification with Uncertainty Quantification](https://mlsanity.com/paper/2607.05393/) (2026)
- [TransNormal-2: Geometry-Grounded Rectified Flow with Edge-Aware Decoding for Precise Normal Estimation](https://mlsanity.com/paper/2609.06665/) (2026)

## Abstract (arXiv)

Single image surface normal estimation and depth estimation are closely related problems as the former can be calculated from the latter. However, the surface normals computed from the output of depth estimation methods are significantly less accurate than the surface normals directly estimated by networks. To reduce such discrepancy, we introduce a novel framework that uses surface normal and its uncertainty to recurrently refine the predicted depth-map. The depth of each pixel can be propagated to a query pixel, using the predicted surface normal as guidance. We thus formulate depth refinement as a classification of choosing the neighboring pixel to propagate from. Then, by propagating to sub-pixel points, we upsample the refined, low-resolution output. The proposed method shows state-of-the-art performance on NYUv2 and iBims-1 - both in terms of depth and normal. Our refinement module can also be attached to the existing depth estimation methods to improve their accuracy. We also show that our framework, only trained for depth estimation, can also be used for depth completion. The code is available at https://github.com/baegwangbin/IronDepth.