# Resilient Sensor Fusion under Adverse Sensor Failures via Multi-Modal Expert Fusion

arXiv: https://arxiv.org/abs/2503.19776 · HTML: https://mlsanity.com/paper/2503.19776/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 37 references that are in the map.
- First posted on arXiv 25 Mar 2025.

## Builds on (most cited first, 30 of 31)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework](https://mlsanity.com/paper/2205.13790/) (2022)
- [SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstruction](https://mlsanity.com/paper/2212.00792/) (2022)
- [CenterMask : Real-Time Anchor-Free Instance Segmentation](https://mlsanity.com/paper/1911.06667/) (2019)
- [FUTR3D: A Unified Sensor Fusion Framework for 3D Detection](https://mlsanity.com/paper/2203.10642/) (2022)
- [Probabilistic and Geometric Depth: Detecting Objects in Perspective](https://mlsanity.com/paper/2107.14160/) (2021)
- [3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection](https://mlsanity.com/paper/2004.12636/) (2020)
- [Robo3D: Towards Robust and Reliable 3D Perception against Corruptions](https://mlsanity.com/paper/2303.17597/) (2023)
- [DeepInteraction: 3D Object Detection via Modality Interaction](https://mlsanity.com/paper/2208.11112/) (2022)
- [Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving](https://mlsanity.com/paper/2303.11040/) (2023)
- [SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds](https://mlsanity.com/paper/2004.02774/) (2020)
- [UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation](https://mlsanity.com/paper/2308.07732/) (2023)
- [Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2205.14951/) (2022)
- [UniBEV: Multi-modal 3D Object Detection with Uniform BEV Encoders for Robustness against Missing Sensor Modalities](https://mlsanity.com/paper/2309.14516/) (2023)

## Built on by (most cited first, 14 of 14)

- [Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategies](https://mlsanity.com/paper/2603.16952/) (2026)
- [Post Fusion Bird's Eye View Feature Stabilization for Robust Multimodal 3D Detection](https://mlsanity.com/paper/2603.05623/) (2026)
- [CCF: Complementary Collaborative Fusion for Domain Generalized Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2603.23276/) (2026)
- [SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions](https://mlsanity.com/paper/2605.11799/) (2026)
- [Can BEV Perception Gracefully Degrade under Sensor Failures?](https://mlsanity.com/paper/2605.30983/) (2026)
- [AnyMod-LLVE: Low-Light Video Enhancement with Modality-Agnostic Inference](https://mlsanity.com/paper/2606.11186/) (2026)
- [RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather](https://mlsanity.com/paper/2607.04587/) (2026)
- [GAUGE: Granularity-Adaptive Counterfactual Gating of Evidence for Incomplete Multimodal Classification](https://mlsanity.com/paper/2608.05608/) (2026)
- [Boundary-Aligned Contribution Routing for Robust Optical--SAR Object Detection](https://mlsanity.com/paper/2608.15261/) (2026)
- [SuppreSensing: Expert-Guided Feature Recalibration and Discrepancy Augmentation for Multimodal Object Detection](https://mlsanity.com/paper/2608.20944/) (2026)
- [SonarLLM: A Native Sonar--Optical Multimodal Large Language Model for Underwater Perception](https://mlsanity.com/paper/2608.24325/) (2026)
- [Variance-Guided Spatial Attention Fusion for Robust End-to-End Driving under Asymmetric Sensor Degradation](https://mlsanity.com/paper/2608.24366/) (2026)
- [SARFusion: Scene-Aware Routing Fusion for Robust Camera-LiDAR 3D Object Detection](https://mlsanity.com/paper/2609.29235/) (2026)
- [SpikeOcc: Towards Resilient Multimodal 3D Occupancy Perception via Spiking Experts](https://mlsanity.com/paper/2610.15745/) (2026)

## Abstract (arXiv)

Modern autonomous driving perception systems utilize complementary multi-modal sensors, such as LiDAR and cameras. Although sensor fusion architectures enhance performance in challenging environments, they still suffer significant performance drops under severe sensor failures, such as LiDAR beam reduction, LiDAR drop, limited field of view, camera drop, and occlusion. This limitation stems from inter-modality dependencies in current sensor fusion frameworks. In this study, we introduce an efficient and robust LiDAR-camera 3D object detector, referred to as MoME, which can achieve robust performance through a mixture of experts approach. Our MoME fully decouples modality dependencies using three parallel expert decoders, which use camera features, LiDAR features, or a combination of both to decode object queries, respectively. We propose Multi-Expert Decoding (MED) framework, where each query is decoded selectively using one of three expert decoders. MoME utilizes an Adaptive Query Router (AQR) to select the most appropriate expert decoder for each query based on the quality of camera and LiDAR features. This ensures that each query is processed by the best-suited expert, resulting in robust performance across diverse sensor failure scenarios. We evaluated the performance of MoME on the nuScenes-R benchmark. Our MoME achieved state-of-the-art performance in extreme weather and sensor failure conditions, significantly outperforming the existing models across various sensor failure scenarios.