# 3D Universal Lesion Detection and Tagging in CT with Self-Training

arXiv: https://arxiv.org/abs/2504.05201 · HTML: https://mlsanity.com/paper/2504.05201/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 7 Apr 2025.

## Builds on (most cited first, 14 of 14)

- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [VarifocalNet: An IoU-aware Dense Object Detector](https://mlsanity.com/paper/2008.13367/) (2020)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [A New 2.5D Representation for Lymph Node Detection using Random Sets of Deep Convolutional Neural Network Observations](https://mlsanity.com/paper/1406.2639/) (2014)
- [Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT](https://mlsanity.com/paper/2009.02577/) (2020)
- [Improving RetinaNet for CT Lesion Detection with Dense Masks from Weak RECIST Labels](https://mlsanity.com/paper/1906.02283/) (2019)
- [3D Context Enhanced Region-based Convolutional Neural Network for End-to-End Lesion Detection](https://mlsanity.com/paper/1806.09648/) (2018)
- [ULDor: A Universal Lesion Detector for CT Scans with Pseudo Masks and Hard Negative Example Mining](https://mlsanity.com/paper/1901.06359/) (2019)
- [SATr: Slice Attention with Transformer for Universal Lesion Detection](https://mlsanity.com/paper/2203.07373/) (2022)
- [Asymmetric 3D Context Fusion for Universal Lesion Detection](https://mlsanity.com/paper/2109.08684/) (2021)
- [AlignShift: Bridging the Gap of Imaging Thickness in 3D Anisotropic Volumes](https://mlsanity.com/paper/2005.01969/) (2020)
- [Lesion Harvester: Iteratively Mining Unlabeled Lesions and Hard-Negative Examples at Scale](https://mlsanity.com/paper/2001.07776/) (2020)
- [Universal Lesion Detection in CT Scans using Neural Network Ensembles](https://mlsanity.com/paper/2111.04886/) (2021)
- [RECIST-Net: Lesion detection via grouping keypoints on RECIST-based annotation](https://mlsanity.com/paper/2107.08715/) (2021)

## Abstract (arXiv)

Radiologists routinely perform the tedious task of lesion localization, classification, and size measurement in computed tomography (CT) studies. Universal lesion detection and tagging (ULDT) can simultaneously help alleviate the cumbersome nature of lesion measurement and enable tumor burden assessment. Previous ULDT approaches utilize the publicly available DeepLesion dataset, however it does not provide the full volumetric (3D) extent of lesions and also displays a severe class imbalance. In this work, we propose a self-training pipeline to detect 3D lesions and tag them according to the body part they occur in. We used a significantly limited 30% subset of DeepLesion to train a VFNet model for 2D lesion detection and tagging. Next, the 2D lesion context was expanded into 3D, and the mined 3D lesion proposals were integrated back into the baseline training data in order to retrain the model over multiple rounds. Through the self-training procedure, our VFNet model learned from its own predictions, detected lesions in 3D, and tagged them. Our results indicated that our VFNet model achieved an average sensitivity of 46.9% at [0.125:8] false positives (FP) with a limited 30% data subset in comparison to the 46.8% of an existing approach that used the entire DeepLesion dataset. To our knowledge, we are the first to jointly detect lesions in 3D and tag them according to the body part label.