# RadSegNet: A Reliable Approach to Radar Camera Fusion

arXiv: https://arxiv.org/abs/2208.03849 · HTML: https://mlsanity.com/paper/2208.03849/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset](https://mlsanity.com/paper/1909.01300/) (2019)
- [RADIATE: A Radar Dataset for Automotive Perception in Bad Weather](https://mlsanity.com/paper/2010.09076/) (2020)
- [A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection](https://mlsanity.com/paper/2005.07431/) (2020)
- [Physics-Based Rendering for Improving Robustness to Rain](https://mlsanity.com/paper/1908.10335/) (2019)
- [Distant Vehicle Detection Using Radar and Vision](https://mlsanity.com/paper/1901.10951/) (2019)
- [RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles](https://mlsanity.com/paper/1905.00526/) (2019)
- [Pointillism: Accurate 3D bounding box estimation with multi-radars](https://mlsanity.com/paper/2203.04440/) (2022)
- [YOdar: Uncertainty-based Sensor Fusion for Vehicle Detection with Camera and Radar Sensors](https://mlsanity.com/paper/2010.03320/) (2020)
- [Radar-Camera Sensor Fusion for Joint Object Detection and Distance Estimation in Autonomous Vehicles](https://mlsanity.com/paper/2009.08428/) (2020)
- [Warping of Radar Data into Camera Image for Cross-Modal Supervision in Automotive Applications](https://mlsanity.com/paper/2012.12809/) (2020)

## Built on by (most cited first, 11 of 11)

- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmarks for Autonomous Driving on Water Surfaces](https://mlsanity.com/paper/2307.06505/) (2023)
- [DPFT: Dual Perspective Fusion Transformer for Camera-Radar-based Object Detection](https://mlsanity.com/paper/2404.03015/) (2024)
- [TransRadar: Adaptive-Directional Transformer for Real-Time Multi-View Radar Semantic Segmentation](https://mlsanity.com/paper/2310.02260/) (2023)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [UniBEVFusion: Unified Radar-Vision BEVFusion for 3D Object Detection](https://mlsanity.com/paper/2409.14751/) (2024)
- [RadarFormer: Lightweight and Accurate Real-Time Radar Object Detection Model](https://mlsanity.com/paper/2304.08447/) (2023)
- [CaRaFFusion: Improving 2D Semantic Segmentation with Camera-Radar Point Cloud Fusion and Zero-Shot Image Inpainting](https://mlsanity.com/paper/2505.03679/) (2025)
- [Synthesizing and Identifying Noise Levels in Autonomous Vehicle Camera Radar Datasets](https://mlsanity.com/paper/2505.00584/) (2025)
- [RadarXFormer: Robust Object Detection via Cross-Dimension Fusion of 4D Radar Spectra and Images for Autonomous Driving](https://mlsanity.com/paper/2603.14822/) (2026)
- [Learning Structurally Consistent Representations for Multi-View Radar Semantic Segmentation](https://mlsanity.com/paper/2606.31609/) (2026)

## Abstract (arXiv)

Perception systems for autonomous driving have seen significant advancements in their performance over last few years. However, these systems struggle to show robustness in extreme weather conditions because sensors like lidars and cameras, which are the primary sensors in a sensor suite, see a decline in performance under these conditions. In order to solve this problem, camera-radar fusion systems provide a unique opportunity for all weather reliable high quality perception. Cameras provides rich semantic information while radars can work through occlusions and in all weather conditions. In this work, we show that the state-of-the-art fusion methods perform poorly when camera input is degraded, which essentially results in losing the all-weather reliability they set out to achieve. Contrary to these approaches, we propose a new method, RadSegNet, that uses a new design philosophy of independent information extraction and truly achieves reliability in all conditions, including occlusions and adverse weather. We develop and validate our proposed system on the benchmark Astyx dataset and further verify these results on the RADIATE dataset. When compared to state-of-the-art methods, RadSegNet achieves a 27% improvement on Astyx and 41.46% increase on RADIATE, in average precision score and maintains a significantly better performance in adverse weather conditions