# Optimal Sensor Data Fusion Architecture for Object Detection in Adverse Weather Conditions

arXiv: https://arxiv.org/abs/1807.02323 · HTML: https://mlsanity.com/paper/1807.02323/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2018.

## Built on by (most cited first, 11 of 11)

- [DSOR: A Scalable Statistical Filter for Removing Falling Snow from LiDAR Point Clouds in Severe Winter Weather](https://mlsanity.com/paper/2109.07078/) (2021)
- [Survey on LiDAR Perception in Adverse Weather Conditions](https://mlsanity.com/paper/2304.06312/) (2023)
- [Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal](https://mlsanity.com/paper/2306.09008/) (2023)
- [An Unsupervised Domain Adaptive Approach for Multimodal 2D Object Detection in Adverse Weather Conditions](https://mlsanity.com/paper/2203.03568/) (2022)
- [Pay "Attention" to Adverse Weather: Weather-aware Attention-based Object Detection](https://mlsanity.com/paper/2204.10803/) (2022)
- [Vulnerable Road User Detection and Safety Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2405.19202/) (2024)
- [DeepQTest: Testing Autonomous Driving Systems with Reinforcement Learning and Real-world Weather Data](https://mlsanity.com/paper/2310.05170/) (2023)
- [PAN: Pillars-Attention-Based Network for 3D Object Detection](https://mlsanity.com/paper/2509.15935/) (2025)
- [GenRadar: Self-supervised Probabilistic Camera Synthesis based on Radar Frequencies](https://mlsanity.com/paper/2107.08948/) (2021)
- [Robust Vehicle Localization and Tracking in Rain using Street Maps](https://mlsanity.com/paper/2409.01038/) (2024)
- [GraphBEV++: Multi-Modal Feature Alignment for Autonomous Driving](https://mlsanity.com/paper/2606.16354/) (2026)

## Abstract (arXiv)

A good and robust sensor data fusion in diverse weather conditions is a quite challenging task. There are several fusion architectures in the literature, e.g. the sensor data can be fused right at the beginning (Early Fusion), or they can be first processed separately and then concatenated later (Late Fusion). In this work, different fusion architectures are compared and evaluated by means of object detection tasks, in which the goal is to recognize and localize predefined objects in a stream of data. Usually, state-of-the-art object detectors based on neural networks are highly optimized for good weather conditions, since the well-known benchmarks only consist of sensor data recorded in optimal weather conditions. Therefore, the performance of these approaches decreases enormously or even fails in adverse weather conditions. In this work, different sensor fusion architectures are compared for good and adverse weather conditions for finding the optimal fusion architecture for diverse weather situations. A new training strategy is also introduced such that the performance of the object detector is greatly enhanced in adverse weather scenarios or if a sensor fails. Furthermore, the paper responds to the question if the detection accuracy can be increased further by providing the neural network with a-priori knowledge such as the spatial calibration of the sensors.