# Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection

arXiv: https://arxiv.org/abs/1902.05291 · HTML: https://mlsanity.com/paper/1902.05291/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2019.

## Built on by (most cited first, 9 of 9)

- [DAMSDet: Dynamic Adaptive Multispectral Detection Transformer with Competitive Query Selection and Adaptive Feature Fusion](https://mlsanity.com/paper/2403.00326/) (2024)
- [Weakly Aligned Feature Fusion for Multimodal Object Detection](https://mlsanity.com/paper/2204.09848/) (2022)
- [Removal then Selection: A Coarse-to-Fine Fusion Perspective for RGB-Infrared Object Detection](https://mlsanity.com/paper/2401.10731/) (2024)
- [FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation](https://mlsanity.com/paper/2103.00738/) (2021)
- [MS-DETR: Multispectral Pedestrian Detection Transformer with Loosely Coupled Fusion and Modality-Balanced Optimization](https://mlsanity.com/paper/2302.00290/) (2023)
- [Pedestrian Detection in Low-Light Conditions: A Comprehensive Survey](https://mlsanity.com/paper/2401.07801/) (2024)
- [Robust pedestrian detection in thermal imagery using synthesized images](https://mlsanity.com/paper/2102.02005/) (2021)
- [DPDETR: Decoupled Position Detection Transformer for Infrared-Visible Object Detection](https://mlsanity.com/paper/2408.06123/) (2024)
- [Partially fake it till you make it: mixing real and fake thermal images for improved object detection](https://mlsanity.com/paper/2106.13603/) (2021)

## Abstract (arXiv)

Effective fusion of complementary information captured by multi-modal sensors (visible and infrared cameras) enables robust pedestrian detection under various surveillance situations (e.g. daytime and nighttime). In this paper, we present a novel box-level segmentation supervised learning framework for accurate and real-time multispectral pedestrian detection by incorporating features extracted in visible and infrared channels. Specifically, our method takes pairs of aligned visible and infrared images with easily obtained bounding box annotations as input and estimates accurate prediction maps to highlight the existence of pedestrians. It offers two major advantages over the existing anchor box based multispectral detection methods. Firstly, it overcomes the hyperparameter setting problem occurred during the training phase of anchor box based detectors and can obtain more accurate detection results, especially for small and occluded pedestrian instances. Secondly, it is capable of generating accurate detection results using small-size input images, leading to improvement of computational efficiency for real-time autonomous driving applications. Experimental results on KAIST multispectral dataset show that our proposed method outperforms state-of-the-art approaches in terms of both accuracy and speed.