# Towards Self-Supervised High Level Sensor Fusion

arXiv: https://arxiv.org/abs/1902.04272 · HTML: https://mlsanity.com/paper/1902.04272/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2019.

## Built on by (most cited first, 1 of 1)

- [GEM: Glare or Gloom, I Can Still See You -- End-to-End Multimodal Object Detection](https://mlsanity.com/paper/2102.12319/) (2021)

## Abstract (arXiv)

In this paper, we present a framework to control a self-driving car by fusing raw information from RGB images and depth maps. A deep neural network architecture is used for mapping the vision and depth information, respectively, to steering commands. This fusion of information from two sensor sources allows to provide redundancy and fault tolerance in the presence of sensor failures. Even if one of the input sensors fails to produce the correct output, the other functioning sensor would still be able to maneuver the car. Such redundancy is crucial in the critical application of self-driving cars. The experimental results have showed that our method is capable of learning to use the relevant sensor information even when one of the sensors fail without any explicit signal.