# Let Video Teaches You More: Video-to-Image Knowledge Distillation using DEtection TRansformer for Medical Video Lesion Detection

arXiv: https://arxiv.org/abs/2408.14051 · HTML: https://mlsanity.com/paper/2408.14051/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- Builds on 16 of its 23 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning](https://mlsanity.com/paper/2201.04676/) (2022)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Memory Enhanced Global-Local Aggregation for Video Object Detection](https://mlsanity.com/paper/2003.12063/) (2020)
- [Accelerating DETR Convergence via Semantic-Aligned Matching](https://mlsanity.com/paper/2203.06883/) (2022)
- [End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2105.10920/) (2021)
- [Distilling Object Detectors with Task Adaptive Regularization](https://mlsanity.com/paper/2006.13108/) (2020)
- [DETRDistill: A Universal Knowledge Distillation Framework for DETR-families](https://mlsanity.com/paper/2211.10156/) (2022)
- [A New Dataset and A Baseline Model for Breast Lesion Detection in Ultrasound Videos](https://mlsanity.com/paper/2207.00141/) (2022)
- [Multi-frame Collaboration for Effective Endoscopic Video Polyp Detection via Spatial-Temporal Feature Transformation](https://mlsanity.com/paper/2107.03609/) (2021)
- [YONA: You Only Need One Adjacent Reference-frame for Accurate and Fast Video Polyp Detection](https://mlsanity.com/paper/2306.03686/) (2023)
- [ECC-PolypDet: Enhanced CenterNet with Contrastive Learning for Automatic Polyp Detection](https://mlsanity.com/paper/2401.04961/) (2024)

## Built on by (most cited first, 1 of 1)

- [Frontiers in Intelligent Colonoscopy](https://mlsanity.com/paper/2410.17241/) (2024)

## Abstract (arXiv)

AI-assisted lesion detection models play a crucial role in the early screening of cancer. However, previous image-based models ignore the inter-frame contextual information present in videos. On the other hand, video-based models capture the inter-frame context but are computationally expensive. To mitigate this contradiction, we delve into Video-to-Image knowledge distillation leveraging DEtection TRansformer (V2I-DETR) for the task of medical video lesion detection. V2I-DETR adopts a teacher-student network paradigm. The teacher network aims at extracting temporal contexts from multiple frames and transferring them to the student network, and the student network is an image-based model dedicated to fast prediction in inference. By distilling multi-frame contexts into a single frame, the proposed V2I-DETR combines the advantages of utilizing temporal contexts from video-based models and the inference speed of image-based models. Through extensive experiments, V2I-DETR outperforms previous state-of-the-art methods by a large margin while achieving the real-time inference speed (30 FPS) as the image-based model.