# Robust Human Detection under Visual Degradation via Thermal and mmWave Radar Fusion

arXiv: https://arxiv.org/abs/2307.03623 · HTML: https://mlsanity.com/paper/2307.03623/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 50 references that are in the map.
- First posted on arXiv 7 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [Getting to Know Low-light Images with The Exclusively Dark Dataset](https://mlsanity.com/paper/1805.11227/) (2018)
- [A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection](https://mlsanity.com/paper/2005.07431/) (2020)
- [Distant Vehicle Detection Using Radar and Vision](https://mlsanity.com/paper/1901.10951/) (2019)
- [2D Car Detection in Radar Data with PointNets](https://mlsanity.com/paper/1904.08414/) (2019)
- [Improved Orientation Estimation and Detection with Hybrid Object Detection Networks for Automotive Radar](https://mlsanity.com/paper/2205.02111/) (2022)
- [Bayesian Triplet Loss: Uncertainty Quantification in Image Retrieval](https://mlsanity.com/paper/2011.12663/) (2020)
- [CramNet: Camera-Radar Fusion with Ray-Constrained Cross-Attention for Robust 3D Object Detection](https://mlsanity.com/paper/2210.09267/) (2022)
- [Pedestrian Detection in Thermal Images using Saliency Maps](https://mlsanity.com/paper/1904.06859/) (2019)
- [UncertaintyFuseNet: Robust Uncertainty-aware Hierarchical Feature Fusion Model with Ensemble Monte Carlo Dropout for COVID-19 Detection](https://mlsanity.com/paper/2105.08590/) (2021)
- [Deep Learning on Radar Centric 3D Object Detection](https://mlsanity.com/paper/2003.00851/) (2020)
- [Scene-aware Learning Network for Radar Object Detection](https://mlsanity.com/paper/2107.01469/) (2021)

## Built on by (most cited first, 3 of 3)

- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [ThermoHands: A Benchmark for 3D Hand Pose Estimation from Egocentric Thermal Images](https://mlsanity.com/paper/2403.09871/) (2024)
- [Non-contact Multimodal Indoor Human Monitoring Systems: A Survey](https://mlsanity.com/paper/2312.07601/) (2023)

## Abstract (arXiv)

The majority of human detection methods rely on the sensor using visible lights (e.g., RGB cameras) but such sensors are limited in scenarios with degraded vision conditions. In this paper, we present a multimodal human detection system that combines portable thermal cameras and single-chip mmWave radars. To mitigate the noisy detection features caused by the low contrast of thermal cameras and the multi-path noise of radar point clouds, we propose a Bayesian feature extractor and a novel uncertainty-guided fusion method that surpasses a variety of competing methods, either single-modal or multi-modal. We evaluate the proposed method on real-world data collection and demonstrate that our approach outperforms the state-of-the-art methods by a large margin.