# Ranking-based Adaptive Query Generation for DETRs in Crowded Pedestrian Detection

arXiv: https://arxiv.org/abs/2310.15725 · HTML: https://mlsanity.com/paper/2310.15725/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 37 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [CityPersons: A Diverse Dataset for Pedestrian Detection](https://mlsanity.com/paper/1702.05693/) (2017)
- [Dense Distinct Query for End-to-End Object Detection](https://mlsanity.com/paper/2303.12776/) (2023)
- [Repulsion Loss: Detecting Pedestrians in a Crowd](https://mlsanity.com/paper/1711.07752/) (2017)
- [Detection in Crowded Scenes: One Proposal, Multiple Predictions](https://mlsanity.com/paper/2003.09163/) (2020)
- [Visual Recognition with Deep Nearest Centroids](https://mlsanity.com/paper/2209.07383/) (2022)
- [Towards a Deeper Understanding of Skeleton-based Gait Recognition](https://mlsanity.com/paper/2204.07855/) (2022)
- [What Makes for End-to-End Object Detection?](https://mlsanity.com/paper/2012.05780/) (2020)
- [NMS by Representative Region: Towards Crowded Pedestrian Detection by Proposal Pairing](https://mlsanity.com/paper/2003.12729/) (2020)
- [Progressive End-to-End Object Detection in Crowded Scenes](https://mlsanity.com/paper/2203.07669/) (2022)
- [PedHunter: Occlusion Robust Pedestrian Detector in Crowded Scenes](https://mlsanity.com/paper/1909.06826/) (2019)
- [From Handcrafted to Deep Features for Pedestrian Detection: A Survey](https://mlsanity.com/paper/2010.00456/) (2020)
- [Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-based Approaches](https://mlsanity.com/paper/2111.06740/) (2021)
- [VLPD: Context-Aware Pedestrian Detection via Vision-Language Semantic Self-Supervision](https://mlsanity.com/paper/2304.03135/) (2023)
- [Variational Pedestrian Detection](https://mlsanity.com/paper/2104.12389/) (2021)
- [One-to-Few Label Assignment for End-to-End Dense Detection](https://mlsanity.com/paper/2303.11567/) (2023)
- [LLA: Loss-aware Label Assignment for Dense Pedestrian Detection](https://mlsanity.com/paper/2101.04307/) (2021)
- [Feature Calibration Network for Occluded Pedestrian Detection](https://mlsanity.com/paper/2212.05717/) (2022)
- [Selecting Learnable Training Samples is All DETRs Need in Crowded Pedestrian Detection](https://mlsanity.com/paper/2305.10801/) (2023)

## Abstract (arXiv)

DEtection TRansformer (DETR) and its variants (DETRs) have been successfully applied to crowded pedestrian detection, which achieved promising performance. However, we find that, in different degrees of crowded scenes, the number of DETRs' queries must be adjusted manually, otherwise, the performance would degrade to varying degrees. In this paper, we first analyze the two current query generation methods and summarize four guidelines for designing the adaptive query generation method. Then, we propose Rank-based Adaptive Query Generation (RAQG) to alleviate the problem. Specifically, we design a rank prediction head that can predict the rank of the lowest confidence positive training sample produced by the encoder. Based on the predicted rank, we design an adaptive selection method that can adaptively select coarse detection results produced by the encoder to generate queries. Moreover, to train the rank prediction head better, we propose Soft Gradient L1 Loss. The gradient of Soft Gradient L1 Loss is continuous, which can describe the relationship between the loss value and the updated value of model parameters granularly. Our method is simple and effective, which can be plugged into any DETRs to make it query-adaptive in theory. The experimental results on Crowdhuman dataset and Citypersons dataset show that our method can adaptively generate queries for DETRs and achieve competitive results. Especially, our method achieves state-of-the-art 39.4% MR on Crowdhuman dataset.