# Surround-View Cameras based Holistic Visual Perception for Automated Driving

arXiv: https://arxiv.org/abs/2206.05542 · HTML: https://mlsanity.com/paper/2206.05542/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 495 references that are in the map.
- First posted on arXiv 11 Jun 2022.

## Builds on (most cited first, 30 of 48)

- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [Exploring Self-attention for Image Recognition](https://mlsanity.com/paper/2004.13621/) (2020)
- [ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1606.02147/) (2016)
- [A study of the effect of JPG compression on adversarial images](https://mlsanity.com/paper/1608.00853/) (2016)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [DSSD : Deconvolutional Single Shot Detector](https://mlsanity.com/paper/1701.06659/) (2017)
- [SfM-Net: Learning of Structure and Motion from Video](https://mlsanity.com/paper/1704.07804/) (2017)
- [Adversarial Objects Against LiDAR-Based Autonomous Driving Systems](https://mlsanity.com/paper/1907.05418/) (2019)
- [Adversarial Transformation Networks: Learning to Generate Adversarial Examples](https://mlsanity.com/paper/1703.09387/) (2017)
- [Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency](https://mlsanity.com/paper/2102.02629/) (2021)
- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [Houdini: Fooling Deep Structured Prediction Models](https://mlsanity.com/paper/1707.05373/) (2017)
- [SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2008.04017/) (2020)
- [Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize](https://mlsanity.com/paper/1707.02937/) (2017)
- [FisheyeDistanceNet: Self-Supervised Scale-Aware Distance Estimation using Monocular Fisheye Camera for Autonomous Driving](https://mlsanity.com/paper/1910.04076/) (2019)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/1406.2661/) (2014)
- [UnRectDepthNet: Self-Supervised Monocular Depth Estimation using a Generic Framework for Handling Common Camera Distortion Models](https://mlsanity.com/paper/2007.06676/) (2020)
- [SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving](https://mlsanity.com/paper/2203.05056/) (2022)
- [A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images](https://mlsanity.com/paper/1912.10230/) (2019)
- [SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras](https://mlsanity.com/paper/2104.04420/) (2021)
- [Challenges in Designing Datasets and Validation for Autonomous Driving](https://mlsanity.com/paper/1901.09270/) (2019)
- [Poly-YOLO: higher speed, more precise detection and instance segmentation for YOLOv3](https://mlsanity.com/paper/2005.13243/) (2020)
- [Weather and Light Level Classification for Autonomous Driving: Dataset, Baseline and Active Learning](https://mlsanity.com/paper/2104.14042/) (2021)

## Built on by (most cited first, 2 of 2)

- [DarSwin: Distortion Aware Radial Swin Transformer](https://mlsanity.com/paper/2304.09691/) (2023)
- [DarSwin-Unet: Distortion Aware Encoder-Decoder Architecture](https://mlsanity.com/paper/2407.17328/) (2024)

## Abstract (arXiv)

The formation of eyes led to the big bang of evolution. The dynamics changed from a primitive organism waiting for the food to come into contact for eating food being sought after by visual sensors. The human eye is one of the most sophisticated developments of evolution, but it still has defects. Humans have evolved a biological perception algorithm capable of driving cars, operating machinery, piloting aircraft, and navigating ships over millions of years. Automating these capabilities for computers is critical for various applications, including self-driving cars, augmented reality, and architectural surveying. Near-field visual perception in the context of self-driving cars can perceive the environment in a range of 0-10 meters and 360deg coverage around the vehicle. It is a critical decision-making component in the development of safer automated driving. Recent advances in computer vision and deep learning, in conjunction with high-quality sensors such as cameras and LiDARs, have fueled mature visual perception solutions. Until now, far-field perception has been the primary focus. Another significant issue is the limited processing power available for developing real-time applications. Because of this bottleneck, there is frequently a trade-off between performance and run-time efficiency. We concentrate on the following issues in order to address them: 1) Developing near-field perception algorithms with high performance and low computational complexity for various visual perception tasks such as geometric and semantic tasks using convolutional neural networks. 2) Using Multi-Task Learning to overcome computational bottlenecks by sharing initial convolutional layers between tasks and developing optimization strategies that balance tasks.