# Optimizing YOLOv5s Object Detection through Knowledge Distillation algorithm

arXiv: https://arxiv.org/abs/2410.12259 · HTML: https://mlsanity.com/paper/2410.12259/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 12 of 12)

- [Survival Prediction Across Diverse Cancer Types Using Neural Networks](https://mlsanity.com/paper/2404.08713/) (2024)
- [Transforming Multidimensional Time Series into Interpretable Event Sequences for Advanced Data Mining](https://mlsanity.com/paper/2409.14327/) (2024)
- [Improving Depth Gradient Continuity in Transformers: A Comparative Study on Monocular Depth Estimation with CNN](https://mlsanity.com/paper/2308.08333/) (2023)
- [Breast Cancer Image Classification Method Based on Deep Transfer Learning](https://mlsanity.com/paper/2404.09226/) (2024)
- [Wasserstein Distance-Weighted Adversarial Network for Cross-Domain Credit Risk Assessment](https://mlsanity.com/paper/2409.18544/) (2024)
- [Research on Image Super-Resolution Reconstruction Mechanism based on Convolutional Neural Network](https://mlsanity.com/paper/2407.13211/) (2024)
- [Contrastive Learning for Knowledge-Based Question Generation in Large Language Models](https://mlsanity.com/paper/2409.13994/) (2024)
- [Applying Hybrid Graph Neural Networks to Strengthen Credit Risk Analysis](https://mlsanity.com/paper/2410.04283/) (2024)
- [Convolutional Neural Networks for Predictive Modeling of Lung Disease](https://mlsanity.com/paper/2408.12605/) (2024)
- [Reducing Bias in Deep Learning Optimization: The RSGDM Approach](https://mlsanity.com/paper/2409.15314/) (2024)
- [Optimizing News Text Classification with Bi-LSTM and Attention Mechanism for Efficient Data Processing](https://mlsanity.com/paper/2409.15576/) (2024)
- [Graph Neural Network Framework for Sentiment Analysis Using Syntactic Feature](https://mlsanity.com/paper/2409.14000/) (2024)

## Built on by (most cited first, 11 of 11)

- [Advanced RAG Models with Graph Structures: Optimizing Complex Knowledge Reasoning and Text Generation](https://mlsanity.com/paper/2411.03572/) (2024)
- [Emotion-Aware Interaction Design in Intelligent User Interface Using Multi-Modal Deep Learning](https://mlsanity.com/paper/2411.06326/) (2024)
- [Metric Learning for Tag Recommendation: Tackling Data Sparsity and Cold Start Issues](https://mlsanity.com/paper/2411.06374/) (2024)
- [Self-Supervised Learning in Deep Networks: A Pathway to Robust Few-Shot Classification](https://mlsanity.com/paper/2411.12151/) (2024)
- [Deep Learning for Cross-Border Transaction Anomaly Detection in Anti-Money Laundering Systems](https://mlsanity.com/paper/2412.07027/) (2024)
- [Reinforcement Learning for Adaptive Resource Scheduling in Complex System Environments](https://mlsanity.com/paper/2411.05346/) (2024)
- [A Combined Encoder and Transformer Approach for Coherent and High-Quality Text Generation](https://mlsanity.com/paper/2411.12157/) (2024)
- [Leveraging Generative Adversarial Networks for Addressing Data Imbalance in Financial Market Supervision](https://mlsanity.com/paper/2412.15222/) (2024)
- [Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models](https://mlsanity.com/paper/2504.13825/) (2025)
- [Architectural Insights into Knowledge Distillation for Object Detection: A Comprehensive Review](https://mlsanity.com/paper/2508.03317/) (2025)
- [DLRMamba: Distilling Low-Rank Mamba for Edge Multispectral Fusion Object Detection](https://mlsanity.com/paper/2603.06920/) (2026)

## Abstract (arXiv)

This paper explores the application of knowledge distillation technology in target detection tasks, especially the impact of different distillation temperatures on the performance of student models. By using YOLOv5l as the teacher network and a smaller YOLOv5s as the student network, we found that with the increase of distillation temperature, the student's detection accuracy gradually improved, and finally achieved mAP50 and mAP50-95 indicators that were better than the original YOLOv5s model at a specific temperature. Experimental results show that appropriate knowledge distillation strategies can not only improve the accuracy of the model but also help improve the reliability and stability of the model in practical applications. This paper also records in detail the accuracy curve and loss function descent curve during the model training process and shows that the model converges to a stable state after 150 training cycles. These findings provide a theoretical basis and technical reference for further optimizing target detection algorithms.