# Learning Lightweight Pedestrian Detector with Hierarchical Knowledge Distillation

arXiv: https://arxiv.org/abs/1909.09325 · HTML: https://mlsanity.com/paper/1909.09325/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 6 of 6)

- [LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector](https://mlsanity.com/paper/2108.08258/) (2021)
- [Object Detection in Autonomous Vehicles: Status and Open Challenges](https://mlsanity.com/paper/2201.07706/) (2022)
- [Visual Exemplar Driven Task-Prompting for Unified Perception in Autonomous Driving](https://mlsanity.com/paper/2303.01788/) (2023)
- [Knowledge Distillation approach towards Melanoma Detection](https://mlsanity.com/paper/2210.08086/) (2022)
- [Improved detection of small objects in road network sequences](https://mlsanity.com/paper/2105.08416/) (2021)
- [A Lightweight Domain Adversarial Neural Network Based on Knowledge Distillation for EEG-based Cross-subject Emotion Recognition](https://mlsanity.com/paper/2305.07446/) (2023)

## Abstract (arXiv)

It remains very challenging to build a pedestrian detection system for real world applications, which demand for both accuracy and speed. This work presents a novel hierarchical knowledge distillation framework to learn a lightweight pedestrian detector, which significantly reduces the computational cost and still holds the high accuracy at the same time. Following the `teacher--student' diagram that a stronger, deeper neural network can teach a lightweight network to learn better representations, we explore multiple knowledge distillation architectures and reframe this approach as a unified, hierarchical distillation framework. In particular, the proposed distillation is performed at multiple hierarchies, multiple stages in a modern detector, which empowers the student detector to learn both low-level details and high-level abstractions simultaneously. Experiment result shows that a student model trained by our framework, with 6 times compression in number of parameters, still achieves competitive performance as the teacher model on the widely used pedestrian detection benchmark.