# HydraFusion: Context-Aware Selective Sensor Fusion for Robust and Efficient Autonomous Vehicle Perception

arXiv: https://arxiv.org/abs/2201.06644 · HTML: https://mlsanity.com/paper/2201.06644/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 17 Jan 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [Expert Gate: Lifelong Learning with a Network of Experts](https://mlsanity.com/paper/1611.06194/) (2016)
- [PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation](https://mlsanity.com/paper/1711.10871/) (2017)
- [Scale-Aware Trident Networks for Object Detection](https://mlsanity.com/paper/1901.01892/) (2019)
- [A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving](https://mlsanity.com/paper/2011.10671/) (2020)
- [RADIATE: A Radar Dataset for Automotive Perception in Bad Weather](https://mlsanity.com/paper/2010.09076/) (2020)
- [A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection](https://mlsanity.com/paper/2005.07431/) (2020)
- [Network of Experts for Large-Scale Image Categorization](https://mlsanity.com/paper/1604.06119/) (2016)
- [Selective Sensor Fusion for Neural Visual-Inertial Odometry](https://mlsanity.com/paper/1903.01534/) (2019)
- [FEEL: Fast, Energy-Efficient Localization for Autonomous Indoor Vehicles](https://mlsanity.com/paper/2102.00702/) (2021)

## Built on by (most cited first, 16 of 16)

- [EcoFusion: Energy-Aware Adaptive Sensor Fusion for Efficient Autonomous Vehicle Perception](https://mlsanity.com/paper/2202.11330/) (2022)
- [Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusion](https://mlsanity.com/paper/2410.12592/) (2024)
- [ContextVLM: Zero-Shot and Few-Shot Context Understanding for Autonomous Driving using Vision Language Models](https://mlsanity.com/paper/2409.00301/) (2024)
- [Hyperdimensional Uncertainty Quantification for Multimodal Uncertainty Fusion in Autonomous Vehicles Perception](https://mlsanity.com/paper/2503.20011/) (2025)
- [Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors](https://mlsanity.com/paper/2407.08049/) (2024)
- [A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion](https://mlsanity.com/paper/2501.07451/) (2025)
- [Romanus: Robust Task Offloading in Modular Multi-Sensor Autonomous Driving Systems](https://mlsanity.com/paper/2207.08865/) (2022)
- [SELF-CARE: Selective Fusion with Context-Aware Low-Power Edge Computing for Stress Detection](https://mlsanity.com/paper/2205.03974/) (2022)
- [METDrive: Multi-modal End-to-end Autonomous Driving with Temporal Guidance](https://mlsanity.com/paper/2409.12667/) (2024)
- [Resource-Efficient Sensor Fusion via System-Wide Dynamic Gated Neural Networks](https://mlsanity.com/paper/2410.16723/) (2024)
- [NaviSlim: Adaptive Context-Aware Navigation and Sensing via Dynamic Slimmable Networks](https://mlsanity.com/paper/2407.01563/) (2024)
- [Performance Implications of Multi-Chiplet Neural Processing Units on Autonomous Driving Perception](https://mlsanity.com/paper/2411.16007/) (2024)
- [CARMA: Context-Aware Runtime Reconfiguration for Energy-Efficient Sensor Fusion](https://mlsanity.com/paper/2306.15748/) (2023)
- [CADENCE: Context-Adaptive Depth Estimation for Navigation and Computational Efficiency](https://mlsanity.com/paper/2604.07286/) (2026)
- [HydraCollab: Adaptive Collaborative-Perception for Distributed Autonomous Systems](https://mlsanity.com/paper/2607.00191/) (2026)
- [CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving](https://mlsanity.com/paper/2608.09202/) (2026)

## Abstract (arXiv)

Although autonomous vehicles (AVs) are expected to revolutionize transportation, robust perception across a wide range of driving contexts remains a significant challenge. Techniques to fuse sensor data from camera, radar, and lidar sensors have been proposed to improve AV perception. However, existing methods are insufficiently robust in difficult driving contexts (e.g., bad weather, low light, sensor obstruction) due to rigidity in their fusion implementations. These methods fall into two broad categories: (i) early fusion, which fails when sensor data is noisy or obscured, and (ii) late fusion, which cannot leverage features from multiple sensors and thus produces worse estimates. To address these limitations, we propose HydraFusion: a selective sensor fusion framework that learns to identify the current driving context and fuses the best combination of sensors to maximize robustness without compromising efficiency. HydraFusion is the first approach to propose dynamically adjusting between early fusion, late fusion, and combinations in-between, thus varying both how and when fusion is applied. We show that, on average, HydraFusion outperforms early and late fusion approaches by 13.66% and 14.54%, respectively, without increasing computational complexity or energy consumption on the industry-standard Nvidia Drive PX2 AV hardware platform. We also propose and evaluate both static and deep-learning-based context identification strategies. Our open-source code and model implementation are available at https://github.com/AICPS/hydrafusion.