# FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera

arXiv: https://arxiv.org/abs/2409.15054 · HTML: https://mlsanity.com/paper/2409.15054/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 30 of 30)

- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras](https://mlsanity.com/paper/1904.04998/) (2019)
- [Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction](https://mlsanity.com/paper/1803.03893/) (2018)
- [Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videos](https://mlsanity.com/paper/1811.06152/) (2018)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2211.13202/) (2022)
- [OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas](https://mlsanity.com/paper/1807.09620/) (2018)
- [Learning monocular depth estimation infusing traditional stereo knowledge](https://mlsanity.com/paper/1904.04144/) (2019)
- [SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes](https://mlsanity.com/paper/2211.03660/) (2022)
- [SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2008.04017/) (2020)
- [Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications](https://mlsanity.com/paper/2004.05560/) (2020)
- [FisheyeDistanceNet: Self-Supervised Scale-Aware Distance Estimation using Monocular Fisheye Camera for Autonomous Driving](https://mlsanity.com/paper/1910.04076/) (2019)
- [UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models](https://mlsanity.com/paper/2007.06676/) (2020)
- [SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras](https://mlsanity.com/paper/2104.04420/) (2021)
- [Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2205.11083/) (2022)
- [Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision](https://mlsanity.com/paper/1803.06192/) (2018)
- [Self-Supervised Monocular Depth Estimation by Direction-aware Cumulative Convolution Network](https://mlsanity.com/paper/2308.05605/) (2023)
- [GasMono: Geometry-Aided Self-Supervised Monocular Depth Estimation for Indoor Scenes](https://mlsanity.com/paper/2309.16019/) (2023)
- [Self-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement](https://mlsanity.com/paper/2302.09789/) (2023)
- [CamLessMonoDepth: Monocular Depth Estimation with Unknown Camera Parameters](https://mlsanity.com/paper/2110.14347/) (2021)
- [RomniStereo: Recurrent Omnidirectional Stereo Matching](https://mlsanity.com/paper/2401.04345/) (2024)
- [OmniSDF: Scene Reconstruction using Omnidirectional Signed Distance Functions and Adaptive Binoctrees](https://mlsanity.com/paper/2404.00678/) (2024)
- [CurbNet: Curb Detection Framework Based on LiDAR Point Cloud Segmentation](https://mlsanity.com/paper/2403.16794/) (2024)
- [OmniColor: A Global Camera Pose Optimization Approach of LiDAR-360Camera Fusion for Colorizing Point Clouds](https://mlsanity.com/paper/2404.04693/) (2024)
- [Estimating Depth of Monocular Panoramic Image with Teacher-Student Model Fusing Equirectangular and Spherical Representations](https://mlsanity.com/paper/2405.16858/) (2024)
- [FSNet: Redesign Self-Supervised MonoDepth for Full-Scale Depth Prediction for Autonomous Driving](https://mlsanity.com/paper/2304.10719/) (2023)

## Built on by (most cited first, 8 of 8)

- [Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens](https://mlsanity.com/paper/2508.04928/) (2025)
- [Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses](https://mlsanity.com/paper/2603.28896/) (2026)
- [Decision-Driven Semantic Object Exploration for Legged Robots via Confidence-Calibrated Perception and Topological Subgoal Selection](https://mlsanity.com/paper/2509.20739/) (2025)
- [RayTun3R: Online Camera Adaptation in 3D Foundation Models](https://mlsanity.com/paper/2607.02711/) (2026)
- [3D Gaussian Splatting with Fisheye Images: Field of View Analysis and Depth-Based Initialization](https://mlsanity.com/paper/2508.06968/) (2025)
- [Geometry-Aware Fisheye-LiDAR Fusion for Robust 3D Object Detection in Low-Overlap Setups](https://mlsanity.com/paper/2606.08844/) (2026)
- [From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation](https://mlsanity.com/paper/2608.27860/) (2026)
- [SFVO: Decoupled Confidence-Guided Stereo-Flow Visual Odometry with Bidirectional PnP](https://mlsanity.com/paper/2609.21754/) (2026)

## Abstract (arXiv)

Accurate depth estimation is crucial for 3D scene comprehension in robotics and autonomous vehicles. Fisheye cameras, known for their wide field of view, have inherent geometric benefits. However, their use in depth estimation is restricted by a scarcity of ground truth data and image distortions. We present FisheyeDepth, a self-supervised depth estimation model tailored for fisheye cameras. We incorporate a fisheye camera model into the projection and reprojection stages during training to handle image distortions, thereby improving depth estimation accuracy and training stability. Furthermore, we incorporate real-scale pose information into the geometric projection between consecutive frames, replacing the poses estimated by the conventional pose network. Essentially, this method offers the necessary physical depth for robotic tasks, and also streamlines the training and inference procedures. Additionally, we devise a multi-channel output strategy to improve robustness by adaptively fusing features at various scales, which reduces the noise from real pose data. We demonstrate the superior performance and robustness of our model in fisheye image depth estimation through evaluations on public datasets and real-world scenarios. The project website is available at: https://github.com/guoyangzhao/FisheyeDepth.