# DeepLesion: Automated Deep Mining, Categorization and Detection of Significant Radiology Image Findings using Large-Scale Clinical Lesion Annotations

arXiv: https://arxiv.org/abs/1710.01766 · HTML: https://mlsanity.com/paper/1710.01766/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2017.

## Built on by (most cited first, 24 of 24)

- [Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning](https://mlsanity.com/paper/2506.07044/) (2025)
- [MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicine](https://mlsanity.com/paper/2408.02900/) (2024)
- [Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback](https://mlsanity.com/paper/2501.01377/) (2025)
- [nnInteractive: Redefining 3D Promptable Segmentation](https://mlsanity.com/paper/2503.08373/) (2025)
- [MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration](https://mlsanity.com/paper/2506.19835/) (2025)
- [Large-scale Long-tailed Disease Diagnosis on Radiology Images](https://mlsanity.com/paper/2312.16151/) (2023)
- [Large Language Model Benchmarks in Medical Tasks](https://mlsanity.com/paper/2410.21348/) (2024)
- [Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data](https://mlsanity.com/paper/2510.14831/) (2025)
- [LesionLocator: Zero-Shot Universal Tumor Segmentation and Tracking in 3D Whole-Body Imaging](https://mlsanity.com/paper/2502.20985/) (2025)
- [On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines](https://mlsanity.com/paper/2405.20459/) (2024)
- [Quad-Net: Quad-domain Network for CT Metal Artifact Reduction](https://mlsanity.com/paper/2207.11678/) (2022)
- [Learning Inductive Attention Guidance for Partially Supervised Pancreatic Ductal Adenocarcinoma Prediction](https://mlsanity.com/paper/2105.14773/) (2021)
- [Visual Textualization for Image Prompted Object Detection](https://mlsanity.com/paper/2506.23785/) (2025)
- [One VLM, Two Roles: Stage-Wise Routing and Specialty-Level Deployment for Clinical Workflows](https://mlsanity.com/paper/2508.16839/) (2025)
- [CT-Bench: A Benchmark for Multimodal Lesion Understanding in Computed Tomography](https://mlsanity.com/paper/2602.14879/) (2026)
- [SE3D: A Framework For Saliency Method Evaluation In 3D Imaging](https://mlsanity.com/paper/2405.14584/) (2024)
- [Explaining 3D Computed Tomography Classifiers with Counterfactuals](https://mlsanity.com/paper/2502.07156/) (2025)
- [Text Embedded Swin-UMamba for DeepLesion Segmentation](https://mlsanity.com/paper/2508.06453/) (2025)
- [State of Abdominal CT Datasets: A Critical Review of Bias, Clinical Relevance, and Real-world Applicability](https://mlsanity.com/paper/2508.13626/) (2025)
- [PolyReal: A Benchmark for Real-World Polymer Science Workflows](https://mlsanity.com/paper/2604.02934/) (2026)
- [Non-negative Subspace Feature Representation for Few-shot Learning in Medical Imaging](https://mlsanity.com/paper/2404.02656/) (2024)
- [MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation](https://mlsanity.com/paper/2511.13135/) (2025)
- [Balancing Efficiency and Restoration: Lightweight Mamba-Based Model for CT Metal Artifact Reduction](https://mlsanity.com/paper/2604.06622/) (2026)
- [Scaling 3D Visual Grounding in Abdominal CT](https://mlsanity.com/paper/2610.04095/) (2026)

## Abstract (arXiv)

Extracting, harvesting and building large-scale annotated radiological image datasets is a greatly important yet challenging problem. It is also the bottleneck to designing more effective data-hungry computing paradigms (e.g., deep learning) for medical image analysis. Yet, vast amounts of clinical annotations (usually associated with disease image findings and marked using arrows, lines, lesion diameters, segmentation, etc.) have been collected over several decades and stored in hospitals' Picture Archiving and Communication Systems. In this paper, we mine and harvest one major type of clinical annotation data - lesion diameters annotated on bookmarked images - to learn an effective multi-class lesion detector via unsupervised and supervised deep Convolutional Neural Networks (CNN). Our dataset is composed of 33,688 bookmarked radiology images from 10,825 studies of 4,477 unique patients. For every bookmarked image, a bounding box is created to cover the target lesion based on its measured diameters. We categorize the collection of lesions using an unsupervised deep mining scheme to generate clustered pseudo lesion labels. Next, we adopt a regional-CNN method to detect lesions of multiple categories, regardless of missing annotations (normally only one lesion is annotated, despite the presence of multiple co-existing findings). Our integrated mining, categorization and detection framework is validated with promising empirical results, as a scalable, universal or multi-purpose CAD paradigm built upon abundant retrospective medical data. Furthermore, we demonstrate that detection accuracy can be significantly improved by incorporating pseudo lesion labels (e.g., Liver lesion/tumor, Lung nodule/tumor, Abdomen lesions, Chest lymph node and others). This dataset will be made publicly available (under the open science initiative).