# Illuminating Pedestrians via Simultaneous Detection & Segmentation

arXiv: https://arxiv.org/abs/1706.08564 · HTML: https://mlsanity.com/paper/1706.08564/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2017.

## Built on by (most cited first, 26 of 26)

- [GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.17202/) (2021)
- [4D-Net for Learned Multi-Modal Alignment](https://mlsanity.com/paper/2109.01066/) (2021)
- [Spatio-Contextual Deep Network Based Multimodal Pedestrian Detection For Autonomous Driving](https://mlsanity.com/paper/2105.12713/) (2021)
- [Fully Understanding Generic Objects: Modeling, Segmentation, and Reconstruction](https://mlsanity.com/paper/2104.00858/) (2021)
- [Exploring Visual Context for Weakly Supervised Person Search](https://mlsanity.com/paper/2106.10506/) (2021)
- [Degrade is Upgrade: Learning Degradation for Low-light Image Enhancement](https://mlsanity.com/paper/2103.10621/) (2021)
- [MPDIoU: A Loss for Efficient and Accurate Bounding Box Regression](https://mlsanity.com/paper/2307.07662/) (2023)
- [Pedestrian Detection: Domain Generalization, CNNs, Transformers and Beyond](https://mlsanity.com/paper/2201.03176/) (2022)
- [NMS-Loss: Learning with Non-Maximum Suppression for Crowded Pedestrian Detection](https://mlsanity.com/paper/2106.02426/) (2021)
- [Deep Learning based Computer Vision Methods for Complex Traffic Environments Perception: A Review](https://mlsanity.com/paper/2211.05120/) (2022)
- [Forecasting Future Instance Segmentation with Learned Optical Flow and Warping](https://mlsanity.com/paper/2211.08049/) (2022)
- [Feature Calibration Network for Occluded Pedestrian Detection](https://mlsanity.com/paper/2212.05717/) (2022)
- [Towards an Efficient Semantic Segmentation Method of ID Cards for Verification Systems](https://mlsanity.com/paper/2111.12764/) (2021)
- [Selecting Learnable Training Samples is All DETRs Need in Crowded Pedestrian Detection](https://mlsanity.com/paper/2305.10801/) (2023)
- [Partially fake it till you make it: mixing real and fake thermal images for improved object detection](https://mlsanity.com/paper/2106.13603/) (2021)
- [Pedestrian Detection by Exemplar-Guided Contrastive Learning](https://mlsanity.com/paper/2111.08974/) (2021)
- [RGB-T Object Detection via Group Shuffled Multi-receptive Attention and Multi-modal Supervision](https://mlsanity.com/paper/2405.18955/) (2024)
- [Nighttime Pedestrian Detection Based on Fore-Background Contrast Learning](https://mlsanity.com/paper/2408.03030/) (2024)
- [Imagine the Unseen: Occluded Pedestrian Detection via Adversarial Feature Completion](https://mlsanity.com/paper/2405.01311/) (2024)
- [Which to Match? Selecting Consistent GT-Proposal Assignment for Pedestrian Detection](https://mlsanity.com/paper/2103.10091/) (2021)
- [Finding a Needle in a Haystack: Tiny Flying Object Detection in 4K Videos using a Joint Detection-and-Tracking Approach](https://mlsanity.com/paper/2105.08253/) (2021)
- [Semantic Segmentation of Anaemic RBCs Using Multilevel Deep Convolutional Encoder-Decoder Network](https://mlsanity.com/paper/2202.04650/) (2022)
- [CircleNet: Reciprocating Feature Adaptation for Robust Pedestrian Detection](https://mlsanity.com/paper/2212.05691/) (2022)
- [Low-light Pedestrian Detection in Visible and Infrared Image Feeds: Issues and Challenges](https://mlsanity.com/paper/2311.08557/) (2023)
- [FPDIoU Loss: A Loss Function for Efficient Bounding Box Regression of Rotated Object Detection](https://mlsanity.com/paper/2405.09942/) (2024)
- [A PST Algorithm for FPs Suppression in Two-stage CNN Detection Methods](https://mlsanity.com/paper/2406.18553/) (2024)

## Abstract (arXiv)

Pedestrian detection is a critical problem in computer vision with significant impact on safety in urban autonomous driving. In this work, we explore how semantic segmentation can be used to boost pedestrian detection accuracy while having little to no impact on network efficiency. We propose a segmentation infusion network to enable joint supervision on semantic segmentation and pedestrian detection. When placed properly, the additional supervision helps guide features in shared layers to become more sophisticated and helpful for the downstream pedestrian detector. Using this approach, we find weakly annotated boxes to be sufficient for considerable performance gains. We provide an in-depth analysis to demonstrate how shared layers are shaped by the segmentation supervision. In doing so, we show that the resulting feature maps become more semantically meaningful and robust to shape and occlusion. Overall, our simultaneous detection and segmentation framework achieves a considerable gain over the state-of-the-art on the Caltech pedestrian dataset, competitive performance on KITTI, and executes 2x faster than competitive methods.