mlsanityData to 7 Oct 2026

Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges

arXiv 2402.12041 ↗First posted 19 Feb 2024cs.CV eess.IVcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we provide a survey on automotive surround-view fisheye optics, with an emphasis on the impact of optical artifacts on computer vision tasks in autonomous driving and ADAS. The automotive industry has advanced in applying state-of-the-art computer vision to enhance road safety and provide automated driving functionality. When using camera systems on vehicles, there is a particular need for a wide field of view to capture the entire vehicle's surroundings, in areas such as low-speed maneuvering, automated parking, and cocoon sensing. However, one crucial challenge in surround-view cameras is the strong optical aberrations of the fisheye camera, which is an area that has received little attention in the literature. Additionally, a comprehensive dataset is needed for testing safety-critical scenarios in vehicle automation. The industry has turned to simulation as a cost-effective strategy for creating synthetic datasets with surround-view camera imagery. We examine different simulation methods (such as model-driven and data-driven simulations) and discuss the simulators' ability (or lack thereof) to model real-world optical performance. Overall, this paper highlights the optical aberrations in automotive fisheye datasets, and the limitations of optical reality in simulated fisheye datasets, with a focus on computer vision in surround-view optical systems.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 52 of its 140 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

52 of 140 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2017Mask R-CNNcited by 7,259
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  10. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  11. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  12. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
38 more
  1. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  2. 2017Deformable Convolutional Networkscited by 1,481
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  4. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  5. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  6. 2016End to End Learning for Self-Driving Carscited by 1,000
  7. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  8. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  9. 2020Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlationcited by 420
  10. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 340
  11. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  12. 2023A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAScited by 149
  13. 2019WoodScape: A multi-task, multi-camera fisheye dataset for autonomous drivingcited by 142
  14. 2020Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentationcited by 98
  15. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  16. 2019Deep Optics for Monocular Depth Estimation and 3D Object Detectioncited by 65
  17. 2018Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Camerascited by 42
  18. 2022Review on Panoramic Imaging and Its Applications in Scene Understandingcited by 42
  19. 2018Learning to Drive from Simulation without Real World Labelscited by 39
  20. 2022Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challengescited by 33
  21. 2022SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Drivingcited by 32
  22. 2023Curriculum Learning for ab initio Deep Learned Refractive Opticscited by 32
  23. 2021AVA: Adversarial Vignetting Attack against Visual Recognitioncited by 29
  24. 2021Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Camerascited by 24
  25. 2022Fast Two-step Blind Optical Aberration Correctioncited by 20
  26. 2018Deep Depth from Defocus: how can defocus blur improve 3D estimation using dense neural networks?cited by 16
  27. 2023Classification robustness to common optical aberrationscited by 13
  28. 2023F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Drivingcited by 12
  29. 2018Modeling Camera Effects to Improve Visual Learning from Synthetic Datacited by 11
  30. 2021Computer vision in automated parking systems: Design, implementation and challengescited by 9
  31. 2023Optical Flow for Autonomous Driving: Applications, Challenges and Improvementscited by 9
  32. 2023Task-Driven Lens Designcited by 9
  33. 2023Neural Lens Modelingcited by 7
  34. 2023From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Drivingcited by 6
  35. 2021FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Imagescited by 5
  36. 2018Realistic Image Degradation with Measured PSFcited by 4
  37. 2021Spherical formulation of geometric motion segmentation constraints in fisheye camerascited by 4
  38. 2023Windscreen Optical Quality for AI Algorithms: Refractive Power and MTF not Sufficientcited by 3

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Visioncited by 23
  2. 2024Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Camerascited by 22
  3. 2025FishBEV: Distortion-Resilient Bird's Eye View Segmentation with Surround-View Fisheye Camerascited by 3