# UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models

arXiv: https://arxiv.org/abs/2007.06676 · HTML: https://mlsanity.com/paper/2007.06676/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2020.

## Built on by (most cited first, 30 of 33)

- [UniK3D: Universal Camera Monocular 3D Estimation](https://mlsanity.com/paper/2503.16591/) (2025)
- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [Toward Practical Monocular Indoor Depth Estimation](https://mlsanity.com/paper/2112.02306/) (2021)
- [Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges](https://mlsanity.com/paper/2205.13281/) (2022)
- [SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving](https://mlsanity.com/paper/2203.05056/) (2022)
- [SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras](https://mlsanity.com/paper/2104.04420/) (2021)
- [Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2103.17001/) (2021)
- [LiMoSeg: Real-time Bird's Eye View based LiDAR Motion Segmentation](https://mlsanity.com/paper/2111.04875/) (2021)
- [Detecting Adversarial Perturbations in Multi-Task Perception](https://mlsanity.com/paper/2203.01177/) (2022)
- [Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving](https://mlsanity.com/paper/2107.07449/) (2021)
- [Self-Supervised Monocular Depth Estimation by Direction-aware Cumulative Convolution Network](https://mlsanity.com/paper/2308.05605/) (2023)
- [F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Driving](https://mlsanity.com/paper/2303.03651/) (2023)
- [DarSwin: Distortion Aware Radial Swin Transformer](https://mlsanity.com/paper/2304.09691/) (2023)
- [An Online Learning System for Wireless Charging Alignment using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2105.12763/) (2021)
- [A Hybrid Sparse-Dense Monocular SLAM System for Autonomous Driving](https://mlsanity.com/paper/2108.07736/) (2021)
- [FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera](https://mlsanity.com/paper/2409.15054/) (2024)
- [SUB-Depth: Self-distillation and Uncertainty Boosting Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2111.09692/) (2021)
- [ViT-BEVSeg: A Hierarchical Transformer Network for Monocular Birds-Eye-View Segmentation](https://mlsanity.com/paper/2205.15667/) (2022)
- [Exploiting the Distortion-Semantic Interaction in Fisheye Data](https://mlsanity.com/paper/2305.00079/) (2023)
- [FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Images](https://mlsanity.com/paper/2103.00191/) (2021)
- [Neuroevolutionary Multi-objective approaches to Trajectory Prediction in Autonomous Vehicles](https://mlsanity.com/paper/2205.02105/) (2022)
- [Depth360: Self-supervised Learning for Monocular Depth Estimation using Learnable Camera Distortion Model](https://mlsanity.com/paper/2110.10415/) (2021)
- [Wid3R: Wide Field-of-View 3D Reconstruction via Camera Model Conditioning](https://mlsanity.com/paper/2602.05321/) (2026)
- [FisheyeDistill: Self-Supervised Monocular Depth Estimation with Ordinal Distillation for Fisheye Cameras](https://mlsanity.com/paper/2205.02930/) (2022)
- [Woodscape Fisheye Object Detection for Autonomous Driving -- CVPR 2022 OmniCV Workshop Challenge](https://mlsanity.com/paper/2206.12912/) (2022)
- [Rethinking Camera Choice: An Empirical Study on Fisheye Camera Properties in Robotic Manipulation](https://mlsanity.com/paper/2603.02139/) (2026)
- [Surround-View Cameras based Holistic Visual Perception for Automated Driving](https://mlsanity.com/paper/2206.05542/) (2022)
- [Multi-camera Bird's Eye View Perception for Autonomous Driving](https://mlsanity.com/paper/2309.09080/) (2023)
- [DaF-BEVSeg: Distortion-aware Fisheye Camera based Bird's Eye View Segmentation with Occlusion Reasoning](https://mlsanity.com/paper/2404.06352/) (2024)
- [High-Precision Self-Supervised Monocular Depth Estimation with Rich-Resource Prior](https://mlsanity.com/paper/2408.00361/) (2024)

## Abstract (arXiv)

In classical computer vision, rectification is an integral part of multi-view depth estimation. It typically includes epipolar rectification and lens distortion correction. This process simplifies the depth estimation significantly, and thus it has been adopted in CNN approaches. However, rectification has several side effects, including a reduced field of view (FOV), resampling distortion, and sensitivity to calibration errors. The effects are particularly pronounced in case of significant distortion (e.g., wide-angle fisheye cameras). In this paper, we propose a generic scale-aware self-supervised pipeline for estimating depth, euclidean distance, and visual odometry from unrectified monocular videos. We demonstrate a similar level of precision on the unrectified KITTI dataset with barrel distortion comparable to the rectified KITTI dataset. The intuition being that the rectification step can be implicitly absorbed within the CNN model, which learns the distortion model without increasing complexity. Our approach does not suffer from a reduced field of view and avoids computational costs for rectification at inference time. To further illustrate the general applicability of the proposed framework, we apply it to wide-angle fisheye cameras with 190^circ horizontal field of view. The training framework UnRectDepthNet takes in the camera distortion model as an argument and adapts projection and unprojection functions accordingly. The proposed algorithm is evaluated further on the KITTI rectified dataset, and we achieve state-of-the-art results that improve upon our previous work FisheyeDistanceNet. Qualitative results on a distorted test scene video sequence indicate excellent performance https://youtu.be/K6pbx3bU4Ss.