# Generalized Object Detection on Fisheye Cameras for Autonomous Driving: Dataset, Representations and Baseline

arXiv: https://arxiv.org/abs/2012.02124 · HTML: https://mlsanity.com/paper/2012.02124/

## Key facts

- Cited by 41 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 30 of 41)

- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024)
- [Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges](https://mlsanity.com/paper/2205.13281/) (2022)
- [SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving](https://mlsanity.com/paper/2203.05056/) (2022)
- [Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2103.17001/) (2021)
- [LiMoSeg: Real-time Bird's Eye View based LiDAR Motion Segmentation](https://mlsanity.com/paper/2111.04875/) (2021)
- [Detecting Adversarial Perturbations in Multi-Task Perception](https://mlsanity.com/paper/2203.01177/) (2022)
- [Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving](https://mlsanity.com/paper/2107.07449/) (2021)
- [F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Driving](https://mlsanity.com/paper/2303.03651/) (2023)
- [Large-Scale Person Detection and Localization using Overhead Fisheye Cameras](https://mlsanity.com/paper/2307.08252/) (2023)
- [DarSwin: Distortion Aware Radial Swin Transformer](https://mlsanity.com/paper/2304.09691/) (2023)
- [Deep Transfer Learning for Intelligent Vehicle Perception: a Survey](https://mlsanity.com/paper/2306.15110/) (2023)
- [UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations](https://mlsanity.com/paper/2411.15355/) (2024)
- [An Online Learning System for Wireless Charging Alignment using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2105.12763/) (2021)
- [Optical Flow for Autonomous Driving: Applications, Challenges and Improvements](https://mlsanity.com/paper/2301.04422/) (2023)
- [Woodscape Fisheye Semantic Segmentation for Autonomous Driving -- CVPR 2021 OmniCV Workshop Challenge](https://mlsanity.com/paper/2107.08246/) (2021)
- [3DGEER: 3D Gaussian Rendering Made Exact and Efficient for Generic Cameras](https://mlsanity.com/paper/2505.24053/) (2025)
- [ViT-BEVSeg: A Hierarchical Transformer Network for Monocular Birds-Eye-View Segmentation](https://mlsanity.com/paper/2205.15667/) (2022)
- [FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Images](https://mlsanity.com/paper/2103.00191/) (2021)
- [What's Cracking? A Review and Analysis of Deep Learning Methods for Structural Crack Segmentation, Detection and Quantification](https://mlsanity.com/paper/2202.03714/) (2022)
- [Surround-view Fisheye BEV-Perception for Valet Parking: Dataset, Baseline and Distortion-insensitive Multi-task Framework](https://mlsanity.com/paper/2212.04111/) (2022)
- [Real-time Streaming Perception System for Autonomous Driving](https://mlsanity.com/paper/2107.14388/) (2021)
- [Applications of Deep Learning for Top-View Omnidirectional Imaging: A Survey](https://mlsanity.com/paper/2304.08193/) (2023)
- [Adapting CNNs for Fisheye Cameras without Retraining](https://mlsanity.com/paper/2404.08187/) (2024)
- [FishRoPE: Projective Rotary Position Embeddings for Omnidirectional Visual Perception](https://mlsanity.com/paper/2604.10391/) (2026)
- [Ensemble-based Semi-supervised Learning to Improve Noisy Soiling Annotations in Autonomous Driving](https://mlsanity.com/paper/2105.07930/) (2021)
- [Woodscape Fisheye Object Detection for Autonomous Driving -- CVPR 2022 OmniCV Workshop Challenge](https://mlsanity.com/paper/2206.12912/) (2022)
- [2.5D Vehicle Odometry Estimation](https://mlsanity.com/paper/2111.08398/) (2021)
- [SpikiLi: A Spiking Simulation of LiDAR based Real-time Object Detection for Autonomous Driving](https://mlsanity.com/paper/2206.02876/) (2022)
- [DarSwin-Unet: Distortion Aware Encoder-Decoder Architecture](https://mlsanity.com/paper/2407.17328/) (2024)

## Abstract (arXiv)

Object detection is a comprehensively studied problem in autonomous driving. However, it has been relatively less explored in the case of fisheye cameras. The standard bounding box fails in fisheye cameras due to the strong radial distortion, particularly in the image's periphery. We explore better representations like oriented bounding box, ellipse, and generic polygon for object detection in fisheye images in this work. We use the IoU metric to compare these representations using accurate instance segmentation ground truth. We design a novel curved bounding box model that has optimal properties for fisheye distortion models. We also design a curvature adaptive perimeter sampling method for obtaining polygon vertices, improving relative mAP score by 4.9% compared to uniform sampling. Overall, the proposed polygon model improves mIoU relative accuracy by 40.3%. It is the first detailed study on object detection on fisheye cameras for autonomous driving scenarios to the best of our knowledge. The dataset comprising of 10,000 images along with all the object representations ground truth will be made public to encourage further research. We summarize our work in a short video with qualitative results at https://youtu.be/iLkOzvJpL-A.