# Universal Semantic Segmentation for Fisheye Urban Driving Images

arXiv: https://arxiv.org/abs/2002.03736 · HTML: https://mlsanity.com/paper/2002.03736/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2020.

## Built on by (most cited first, 25 of 25)

- [Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation](https://mlsanity.com/paper/2203.01452/) (2022)
- [Capturing Omni-Range Context for Omnidirectional Segmentation](https://mlsanity.com/paper/2103.05687/) (2021)
- [Review on Panoramic Imaging and Its Applications in Scene Understanding](https://mlsanity.com/paper/2205.05570/) (2022)
- [Depth Any Camera: Zero-Shot Metric Depth Estimation from Any Camera](https://mlsanity.com/paper/2501.02464/) (2025)
- [DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange](https://mlsanity.com/paper/2108.06383/) (2021)
- [Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation](https://mlsanity.com/paper/2110.11062/) (2021)
- [Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges](https://mlsanity.com/paper/2205.13281/) (2022)
- [Adaptable Deformable Convolutions for Semantic Segmentation of Fisheye Images in Autonomous Driving Systems](https://mlsanity.com/paper/2102.10191/) (2021)
- [F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Driving](https://mlsanity.com/paper/2303.03651/) (2023)
- [FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognition](https://mlsanity.com/paper/2203.07255/) (2022)
- [DarSwin: Distortion Aware Radial Swin Transformer](https://mlsanity.com/paper/2304.09691/) (2023)
- [Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2407.02182/) (2024)
- [FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation](https://mlsanity.com/paper/2303.13842/) (2023)
- [Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos](https://mlsanity.com/paper/2105.07209/) (2021)
- [ADD: An Automatic Desensitization Fisheye Dataset for Autonomous Driving](https://mlsanity.com/paper/2308.07590/) (2023)
- [Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2506.21198/) (2025)
- [Adapting CNNs for Fisheye Cameras without Retraining](https://mlsanity.com/paper/2404.08187/) (2024)
- [Sector Patch Embedding: An Embedding Module Conforming to The Distortion Pattern of Fisheye Image](https://mlsanity.com/paper/2303.14645/) (2023)
- [Transparency Distortion Robustness for SOTA Image Segmentation Tasks](https://mlsanity.com/paper/2405.12864/) (2024)
- [Deformable Convolution Based Road Scene Semantic Segmentation of Fisheye Images in Autonomous Driving](https://mlsanity.com/paper/2407.16647/) (2024)
- [DarSwin-Unet: Distortion Aware Encoder-Decoder Architecture](https://mlsanity.com/paper/2407.17328/) (2024)
- [Exploring Surround-View Fisheye Camera 3D Object Detection](https://mlsanity.com/paper/2511.18695/) (2025)
- [LineMarkNet: Line Landmark Detection for Valet Parking](https://mlsanity.com/paper/2309.10475/) (2023)
- [Equirectangular image construction method for standard CNNs for Semantic Segmentation](https://mlsanity.com/paper/2310.09122/) (2023)
- [Location-guided Head Pose Estimation for Fisheye Image](https://mlsanity.com/paper/2402.18320/) (2024)

## Abstract (arXiv)

Semantic segmentation is a critical method in the field of autonomous driving. When performing semantic image segmentation, a wider field of view (FoV) helps to obtain more information about the surrounding environment, making automatic driving safer and more reliable, which could be offered by fisheye cameras. However, large public fisheye datasets are not available, and the fisheye images captured by the fisheye camera with large FoV comes with large distortion, so commonly-used semantic segmentation model cannot be directly utilized. In this paper, a seven degrees of freedom (DoF) augmentation method is proposed to transform rectilinear image to fisheye image in a more comprehensive way. In the training process, rectilinear images are transformed into fisheye images in seven DoF, which simulates the fisheye images taken by cameras of different positions, orientations and focal lengths. The result shows that training with the seven-DoF augmentation can improve the model's accuracy and robustness against different distorted fisheye data. This seven-DoF augmentation provides a universal semantic segmentation solution for fisheye cameras in different autonomous driving applications. Also, we provide specific parameter settings of the augmentation for autonomous driving. At last, we tested our universal semantic segmentation model on real fisheye images and obtained satisfactory results. The code and configurations are released at https://github.com/Yaozhuwa/FisheyeSeg.