# Auto-Rectify Network for Unsupervised Indoor Depth Estimation

arXiv: https://arxiv.org/abs/2006.02708 · HTML: https://mlsanity.com/paper/2006.02708/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2020.

## Built on by (most cited first, 30 of 39)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors](https://mlsanity.com/paper/2503.17316/) (2025)
- [SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes](https://mlsanity.com/paper/2211.03660/) (2022)
- [Matrix3D: Large Photogrammetry Model All-in-One](https://mlsanity.com/paper/2502.07685/) (2025)
- [FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models](https://mlsanity.com/paper/2308.05733/) (2023)
- [GasMono: Geometry-Aided Self-Supervised Monocular Depth Estimation for Indoor Scenes](https://mlsanity.com/paper/2309.16019/) (2023)
- [DCPI-Depth: Explicitly Infusing Dense Correspondence Prior to Unsupervised Monocular Depth Estimation](https://mlsanity.com/paper/2405.16960/) (2024)
- [MonoIndoor++:Towards Better Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2207.08951/) (2022)
- [Depth Anything at Any Condition](https://mlsanity.com/paper/2507.01634/) (2025)
- [Towards 3D Scene Reconstruction from Locally Scale-Aligned Monocular Video Depth](https://mlsanity.com/paper/2202.01470/) (2022)
- [Semi-Supervised Learning for Visual Bird's Eye View Semantic Segmentation](https://mlsanity.com/paper/2308.14525/) (2023)
- [Deeper into Self-Supervised Monocular Indoor Depth Estimation](https://mlsanity.com/paper/2312.01283/) (2023)
- [NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimation](https://mlsanity.com/paper/2307.08695/) (2023)
- [SAB3R: Semantic-Augmented Backbone in 3D Reconstruction](https://mlsanity.com/paper/2506.02112/) (2025)
- [StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation](https://mlsanity.com/paper/2506.20756/) (2025)
- [Continual Learning of Unsupervised Monocular Depth from Videos](https://mlsanity.com/paper/2311.02393/) (2023)
- [3D Scene Understanding Through Local Random Access Sequence Modeling](https://mlsanity.com/paper/2504.03875/) (2025)
- [MeSa: Masked, Geometric, and Supervised Pre-training for Monocular Depth Estimation](https://mlsanity.com/paper/2310.04551/) (2023)
- [Meta-Optimization for Higher Model Generalizability in Single-Image Depth Prediction](https://mlsanity.com/paper/2305.07269/) (2023)
- [GAM-Depth: Self-Supervised Indoor Depth Estimation Leveraging a Gradient-Aware Mask and Semantic Constraints](https://mlsanity.com/paper/2402.14354/) (2024)
- [G3Splat: Geometrically Consistent Generalizable Gaussian Splatting](https://mlsanity.com/paper/2512.17547/) (2025)
- [CogniMap3D: Cognitive 3D Mapping and Rapid Retrieval](https://mlsanity.com/paper/2601.08175/) (2026)
- [UNO: Unified Self-Supervised Monocular Odometry for Platform-Agnostic Deployment](https://mlsanity.com/paper/2506.07013/) (2025)
- [Snap-Snap: Taking Two Images to Reconstruct 3D Human Gaussians in Milliseconds](https://mlsanity.com/paper/2508.14892/) (2025)
- [Unified 3D Scene Understanding Through Physical World Modeling](https://mlsanity.com/paper/2605.24321/) (2026)
- [Improving Neural Indoor Surface Reconstruction with Mask-Guided Adaptive Consistency Constraints](https://mlsanity.com/paper/2309.09739/) (2023)
- [InSpaceType: Reconsider Space Type in Indoor Monocular Depth Estimation](https://mlsanity.com/paper/2309.13516/) (2023)
- [Self-Supervised Geometry-Guided Initialization for Robust Monocular Visual Odometry](https://mlsanity.com/paper/2406.00929/) (2024)

## Abstract (arXiv)

Single-View depth estimation using the CNNs trained from unlabelled videos has shown significant promise. However, excellent results have mostly been obtained in street-scene driving scenarios, and such methods often fail in other settings, particularly indoor videos taken by handheld devices. In this work, we establish that the complex ego-motions exhibited in handheld settings are a critical obstacle for learning depth. Our fundamental analysis suggests that the rotation behaves as noise during training, as opposed to the translation (baseline) which provides supervision signals. To address the challenge, we propose a data pre-processing method that rectifies training images by removing their relative rotations for effective learning. The significantly improved performance validates our motivation. Towards end-to-end learning without requiring pre-processing, we propose an Auto-Rectify Network with novel loss functions, which can automatically learn to rectify images during training. Consequently, our results outperform the previous unsupervised SOTA method by a large margin on the challenging NYUv2 dataset. We also demonstrate the generalization of our trained model in ScanNet and Make3D, and the universality of our proposed learning method on 7-Scenes and KITTI datasets.