# Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision

arXiv: https://arxiv.org/abs/1803.06192 · HTML: https://mlsanity.com/paper/1803.06192/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2018.

## Built on by (most cited first, 19 of 19)

- [SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving](https://mlsanity.com/paper/2203.05056/) (2022)
- [SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras](https://mlsanity.com/paper/2104.04420/) (2021)
- [Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2103.17001/) (2021)
- [LiMoSeg: Real-time Bird's Eye View based LiDAR Motion Segmentation](https://mlsanity.com/paper/2111.04875/) (2021)
- [Detecting Adversarial Perturbations in Multi-Task Perception](https://mlsanity.com/paper/2203.01177/) (2022)
- [Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving](https://mlsanity.com/paper/2107.07449/) (2021)
- [An Online Learning System for Wireless Charging Alignment using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2105.12763/) (2021)
- [FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera](https://mlsanity.com/paper/2409.15054/) (2024)
- [LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR](https://mlsanity.com/paper/2109.03569/) (2021)
- [Autonomous Driving Small-Scale Cars: A Survey of Recent Development](https://mlsanity.com/paper/2404.06229/) (2024)
- [Convolution kernel adaptation to calibrated fisheye](https://mlsanity.com/paper/2402.01456/) (2024)
- [Depth360: Self-supervised Learning for Monocular Depth Estimation using Learnable Camera Distortion Model](https://mlsanity.com/paper/2110.10415/) (2021)
- [Woodscape Fisheye Object Detection for Autonomous Driving -- CVPR 2022 OmniCV Workshop Challenge](https://mlsanity.com/paper/2206.12912/) (2022)
- [Ensemble-based Semi-supervised Learning to Improve Noisy Soiling Annotations in Autonomous Driving](https://mlsanity.com/paper/2105.07930/) (2021)
- [Surround-View Cameras based Holistic Visual Perception for Automated Driving](https://mlsanity.com/paper/2206.05542/) (2022)
- [SpikiLi: A Spiking Simulation of LiDAR based Real-time Object Detection for Autonomous Driving](https://mlsanity.com/paper/2206.02876/) (2022)
- [RS3DBench: A Comprehensive Benchmark for 3D Spatial Perception in Remote Sensing](https://mlsanity.com/paper/2509.18897/) (2025)
- [WoodScape Motion Segmentation for Autonomous Driving -- CVPR 2023 OmniCV Workshop Challenge](https://mlsanity.com/paper/2401.00910/) (2023)
- [Neural Rendering based Urban Scene Reconstruction for Autonomous Driving](https://mlsanity.com/paper/2402.06826/) (2024)

## Abstract (arXiv)

Near field depth estimation around a self driving car is an important function that can be achieved by four wide angle fisheye cameras having a field of view of over 180. Depth estimation based on convolutional neural networks (CNNs) produce state of the art results, but progress is hindered because depth annotation cannot be obtained manually. Synthetic datasets are commonly used but they have limitations. For instance, they do not capture the extensive variability in the appearance of objects like vehicles present in real datasets. There is also a domain shift while performing inference on natural images illustrated by many attempts to handle the domain adaptation explicitly. In this work, we explore an alternate approach of training using sparse LiDAR data as ground truth for depth estimation for fisheye camera. We built our own dataset using our self driving car setup which has a 64 beam Velodyne LiDAR and four wide angle fisheye cameras. To handle the difference in view points of LiDAR and fisheye camera, an occlusion resolution mechanism was implemented. We started with Eigen's multiscale convolutional network architecture and improved by modifying activation function and optimizer. We obtained promising results on our dataset with RMSE errors comparable to the state of the art results obtained on KITTI.