# Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges

arXiv: https://arxiv.org/abs/2205.13281 · HTML: https://mlsanity.com/paper/2205.13281/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 168 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 30 of 62)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks](https://mlsanity.com/paper/2003.13402/) (2020)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)
- [Tangent Images for Mitigating Spherical Distortion](https://mlsanity.com/paper/1912.09390/) (2019)
- [Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera](https://mlsanity.com/paper/1803.05959/) (2018)
- [DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications](https://mlsanity.com/paper/1810.12186/) (2018)
- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [Kernel Transformer Networks for Compact Spherical Convolution](https://mlsanity.com/paper/1812.03115/) (2018)
- [The Double Sphere Camera Model](https://mlsanity.com/paper/1807.08957/) (2018)
- [SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2008.04017/) (2020)
- [Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Cameras](https://mlsanity.com/paper/1801.00708/) (2018)
- [Generalized Object Detection on Fisheye Cameras for Autonomous Driving: Dataset, Representations and Baseline](https://mlsanity.com/paper/2012.02124/) (2020)
- [FisheyeDistanceNet: Self-Supervised Scale-Aware Distance Estimation using Monocular Fisheye Camera for Autonomous Driving](https://mlsanity.com/paper/1910.04076/) (2019)
- [Dynamic Task Weighting Methods for Multi-task Networks in Autonomous Driving Systems](https://mlsanity.com/paper/2001.02223/) (2020)
- [UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models](https://mlsanity.com/paper/2007.06676/) (2020)
- [SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving](https://mlsanity.com/paper/2203.05056/) (2022)
- [GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation](https://mlsanity.com/paper/1803.03254/) (2018)
- [Universal Semantic Segmentation for Fisheye Urban Driving Images](https://mlsanity.com/paper/2002.03736/) (2020)
- [SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras](https://mlsanity.com/paper/2104.04420/) (2021)

## Built on by (most cited first, 30 of 34)

- [Polar Parametrization for Vision-based Surround-View 3D Detection](https://mlsanity.com/paper/2206.10965/) (2022)
- [F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Driving](https://mlsanity.com/paper/2303.03651/) (2023)
- [ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modeling](https://mlsanity.com/paper/2603.02697/) (2026)
- [Optical Flow for Autonomous Driving: Applications, Challenges and Improvements](https://mlsanity.com/paper/2301.04422/) (2023)
- [3DGEER: 3D Gaussian Rendering Made Exact and Efficient for Generic Cameras](https://mlsanity.com/paper/2505.24053/) (2025)
- [FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation](https://mlsanity.com/paper/2303.13842/) (2023)
- [Exploiting the Distortion-Semantic Interaction in Fisheye Data](https://mlsanity.com/paper/2305.00079/) (2023)
- [Beyond the Field-of-View: Enhancing Scene Visibility and Perception with Clip-Recurrent Transformer](https://mlsanity.com/paper/2211.11293/) (2022)
- [Surround-view Fisheye BEV-Perception for Valet Parking: Dataset, Baseline and Distortion-insensitive Multi-task Framework](https://mlsanity.com/paper/2212.04111/) (2022)
- [Wid3R: Wide Field-of-View 3D Reconstruction via Camera Model Conditioning](https://mlsanity.com/paper/2602.05321/) (2026)
- [Woodscape Fisheye Object Detection for Autonomous Driving -- CVPR 2022 OmniCV Workshop Challenge](https://mlsanity.com/paper/2206.12912/) (2022)
- [Applications of Deep Learning for Top-View Omnidirectional Imaging: A Survey](https://mlsanity.com/paper/2304.08193/) (2023)
- [BEVSeg2TP: Surround View Camera Bird's-Eye-View Based Joint Vehicle Segmentation and Ego Vehicle Trajectory Prediction](https://mlsanity.com/paper/2312.13081/) (2023)
- [Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges](https://mlsanity.com/paper/2402.12041/) (2024)
- [FishBEV: Distortion-Resilient Bird's Eye View Segmentation with Surround-View Fisheye Cameras](https://mlsanity.com/paper/2509.13681/) (2025)
- [Fisheye Camera and Ultrasonic Sensor Fusion For Near-Field Obstacle Perception in Bird's-Eye-View](https://mlsanity.com/paper/2402.00637/) (2024)
- [DaF-BEVSeg: Distortion-aware Fisheye Camera based Bird's Eye View Segmentation with Occlusion Reasoning](https://mlsanity.com/paper/2404.06352/) (2024)
- [FisheyeDetNet: 360deg Surround view Fisheye Camera based Object Detection System for Autonomous Driving](https://mlsanity.com/paper/2404.13443/) (2024)
- [M"obius Transform for Mitigating Perspective Distortions in Representation Learning](https://mlsanity.com/paper/2405.02296/) (2024)
- [Benchmarking Multi-View BEV Object Detection with Mixed Pinhole and Fisheye Cameras](https://mlsanity.com/paper/2603.27818/) (2026)
- [Measuring Natural Scenes SFR of Automotive Fisheye Cameras](https://mlsanity.com/paper/2401.05232/) (2024)
- [DiffPoint: Single and Multi-view Point Cloud Reconstruction with ViT Based Diffusion Model](https://mlsanity.com/paper/2402.11241/) (2024)
- [MESC-3D:Mining Effective Semantic Cues for 3D Reconstruction from a Single Image](https://mlsanity.com/paper/2502.20861/) (2025)
- [Systematic Comparison of Projection Methods for Monocular 3D Human Pose Estimation on Fisheye Images](https://mlsanity.com/paper/2506.19747/) (2025)
- [FisheyeGaussianLift: BEV Feature Lifting for Surround-View Fisheye Camera Perception](https://mlsanity.com/paper/2511.17210/) (2025)
- [WoodScape Motion Segmentation for Autonomous Driving -- CVPR 2023 OmniCV Workshop Challenge](https://mlsanity.com/paper/2401.00910/) (2023)
- [Neural Rendering based Urban Scene Reconstruction for Autonomous Driving](https://mlsanity.com/paper/2402.06826/) (2024)
- [Impact of Video Compression Artifacts on Fisheye Camera Visual Perception Tasks](https://mlsanity.com/paper/2403.16338/) (2024)
- [Click-Calib: A Robust Extrinsic Calibration Method for Surround-View Systems](https://mlsanity.com/paper/2501.01557/) (2025)
- [A 360-degree Multi-camera System for Blue Emergency Light Detection Using Color Attention RT-DETR and the ABLDataset](https://mlsanity.com/paper/2603.05058/) (2026)

## Abstract (arXiv)

Surround-view fisheye cameras are commonly used for near-field sensing in automated driving. Four fisheye cameras on four sides of the vehicle are sufficient to cover 360deg around the vehicle capturing the entire near-field region. Some primary use cases are automated parking, traffic jam assist, and urban driving. There are limited datasets and very little work on near-field perception tasks as the focus in automotive perception is on far-field perception. In contrast to far-field, surround-view perception poses additional challenges due to high precision object detection requirements of 10cm and partial visibility of objects. Due to the large radial distortion of fisheye cameras, standard algorithms cannot be extended easily to the surround-view use case. Thus, we are motivated to provide a self-contained reference for automotive fisheye camera perception for researchers and practitioners. Firstly, we provide a unified and taxonomic treatment of commonly used fisheye camera models. Secondly, we discuss various perception tasks and existing literature. Finally, we discuss the challenges and future direction.