# Selecting Learnable Training Samples is All DETRs Need in Crowded Pedestrian Detection

arXiv: https://arxiv.org/abs/2305.10801 · HTML: https://mlsanity.com/paper/2305.10801/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 50 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 30 of 31)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [OTA: Optimal Transport Assignment for Object Detection](https://mlsanity.com/paper/2103.14259/) (2021)
- [CityPersons: A Diverse Dataset for Pedestrian Detection](https://mlsanity.com/paper/1702.05693/) (2017)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)
- [TIDE: A General Toolbox for Identifying Object Detection Errors](https://mlsanity.com/paper/2008.08115/) (2020)
- [Repulsion Loss: Detecting Pedestrians in a Crowd](https://mlsanity.com/paper/1711.07752/) (2017)
- [Detection in Crowded Scenes: One Proposal, Multiple Predictions](https://mlsanity.com/paper/2003.09163/) (2020)
- [What Makes for End-to-End Object Detection?](https://mlsanity.com/paper/2012.05780/) (2020)
- [NMS by Representative Region: Towards Crowded Pedestrian Detection by Proposal Pairing](https://mlsanity.com/paper/2003.12729/) (2020)
- [Progressive End-to-End Object Detection in Crowded Scenes](https://mlsanity.com/paper/2203.07669/) (2022)
- [PedHunter: Occlusion Robust Pedestrian Detector in Crowded Scenes](https://mlsanity.com/paper/1909.06826/) (2019)
- [Illuminating Pedestrians via Simultaneous Detection & Segmentation](https://mlsanity.com/paper/1706.08564/) (2017)
- [Pedestrian Detection aided by Deep Learning Semantic Tasks](https://mlsanity.com/paper/1412.0069/) (2014)
- [Pedestrian Detection with Autoregressive Network Phases](https://mlsanity.com/paper/1812.00440/) (2018)
- [Variational Pedestrian Detection](https://mlsanity.com/paper/2104.12389/) (2021)
- [Coupled Network for Robust Pedestrian Detection with Gated Multi-Layer Feature Extraction and Deformable Occlusion Handling](https://mlsanity.com/paper/1912.08661/) (2019)

## Built on by (most cited first, 3 of 3)

- [Crowd-SAM: SAM as a Smart Annotator for Object Detection in Crowded Scenes](https://mlsanity.com/paper/2407.11464/) (2024)
- [Stay in your Lane: Role Specific Queries with Overlap Suppression Loss for Dense Video Captioning](https://mlsanity.com/paper/2603.11439/) (2026)
- [Ranking-based Adaptive Query Generation for DETRs in Crowded Pedestrian Detection](https://mlsanity.com/paper/2310.15725/) (2023)

## Abstract (arXiv)

DEtection TRansformer (DETR) and its variants (DETRs) achieved impressive performance in general object detection. However, in crowded pedestrian detection, the performance of DETRs is still unsatisfactory due to the inappropriate sample selection method which results in more false positives. To settle the issue, we propose a simple but effective sample selection method for DETRs, Sample Selection for Crowded Pedestrians (SSCP), which consists of the constraint-guided label assignment scheme (CGLA) and the utilizability-aware focal loss (UAFL). Our core idea is to select learnable samples for DETRs and adaptively regulate the loss weights of samples based on their utilizability. Specifically, in CGLA, we proposed a new cost function to ensure that only learnable positive training samples are retained and the rest are negative training samples. Further, considering the utilizability of samples, we designed UAFL to adaptively assign different loss weights to learnable positive samples depending on their gradient ratio and IoU. Experimental results show that the proposed SSCP effectively improves the baselines without introducing any overhead in inference. Especially, Iter Deformable DETR is improved to 39.7(-2.0)% MR on Crowdhuman and 31.8(-0.4)% MR on Citypersons.