# Investigating the Effect of Sensor Modalities in Multi-Sensor Detection-Prediction Models

arXiv: https://arxiv.org/abs/2101.03279 · HTML: https://mlsanity.com/paper/2101.03279/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 14 references that are in the map.
- First posted on arXiv 9 Jan 2021.

## Builds on (most cited first, 11 of 11)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Continuous Fusion for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2012.10992/) (2020)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World](https://mlsanity.com/paper/2006.09348/) (2020)
- [Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation](https://mlsanity.com/paper/1904.11466/) (2019)
- [RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects](https://mlsanity.com/paper/2007.14366/) (2020)
- [Multi-View Fusion of Sensor Data for Improved Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2008.11901/) (2020)
- [MultiXNet: Multiclass Multistage Multimodal Motion Prediction](https://mlsanity.com/paper/2006.02000/) (2020)
- [LaserFlow: Efficient and Probabilistic Object Detection and Motion Forecasting](https://mlsanity.com/paper/2003.05982/) (2020)
- [Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation](https://mlsanity.com/paper/1705.10422/) (2017)
- [LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion](https://mlsanity.com/paper/2010.00731/) (2020)

## Built on by (most cited first, 3 of 3)

- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [Robust sensor fusion against on-vehicle sensor staleness](https://mlsanity.com/paper/2506.05780/) (2025)
- [ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning](https://mlsanity.com/paper/2607.09884/) (2026)

## Abstract (arXiv)

Detection of surrounding objects and their motion prediction are critical components of a self-driving system. Recently proposed models that jointly address these tasks rely on a number of sensors to achieve state-of-the-art performance. However, this increases system complexity and may result in a brittle model that overfits to any single sensor modality while ignoring others, leading to reduced generalization. We focus on this important problem and analyze the contribution of sensor modalities towards the model performance. In addition, we investigate the use of sensor dropout to mitigate the above-mentioned issues, leading to a more robust, better-performing model on real-world driving data.