# Knowledge Distillation in YOLOX-ViT for Side-Scan Sonar Object Detection

arXiv: https://arxiv.org/abs/2403.09313 · HTML: https://mlsanity.com/paper/2403.09313/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [Pruning Convolutional Neural Networks for Resource Efficient Inference](https://mlsanity.com/paper/1611.06440/) (2016)
- [Quantization Mimic: Towards Very Tiny CNN for Object Detection](https://mlsanity.com/paper/1805.02152/) (2018)
- [Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Prediction](https://mlsanity.com/paper/1504.03293/) (2015)
- [Assessing The Performance of YOLOv5 Algorithm for Detecting Volunteer Cotton Plants in Corn Fields at Three Different Growth Stages](https://mlsanity.com/paper/2208.00519/) (2022)

## Built on by (most cited first, 3 of 3)

- [Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challenges](https://mlsanity.com/paper/2412.11840/) (2024)
- [Sensing for Space Safety and Sustainability: A Deep Learning Approach with Vision Transformers](https://mlsanity.com/paper/2412.08913/) (2024)
- [ROSAR: An Adversarial Re-Training Framework for Robust Side-Scan Sonar Object Detection](https://mlsanity.com/paper/2410.10554/) (2024)

## Abstract (arXiv)

In this paper we present YOLOX-ViT, a novel object detection model, and investigate the efficacy of knowledge distillation for model size reduction without sacrificing performance. Focused on underwater robotics, our research addresses key questions about the viability of smaller models and the impact of the visual transformer layer in YOLOX. Furthermore, we introduce a new side-scan sonar image dataset, and use it to evaluate our object detector's performance. Results show that knowledge distillation effectively reduces false positives in wall detection. Additionally, the introduced visual transformer layer significantly improves object detection accuracy in the underwater environment. The source code of the knowledge distillation in the YOLOX-ViT is at https://github.com/remaro-network/KD-YOLOX-ViT.