# Deep Lesion Graphs in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-scale Lesion Database

arXiv: https://arxiv.org/abs/1711.10535 · HTML: https://mlsanity.com/paper/1711.10535/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2017.

## Built on by (most cited first, 30 of 42)

- [Explainable artificial intelligence (XAI) in deep learning-based medical image analysis](https://mlsanity.com/paper/2107.10912/) (2021)
- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics](https://mlsanity.com/paper/2111.13424/) (2021)
- [GAIA: A Transfer Learning System of Object Detection that Fits Your Needs](https://mlsanity.com/paper/2106.11346/) (2021)
- [The ULS23 Challenge: a Baseline Model and Benchmark Dataset for 3D Universal Lesion Segmentation in Computed Tomography](https://mlsanity.com/paper/2406.05231/) (2024)
- [DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets](https://mlsanity.com/paper/2311.04894/) (2023)
- [InDuDoNet+: A Deep Unfolding Dual Domain Network for Metal Artifact Reduction in CT Images](https://mlsanity.com/paper/2112.12660/) (2021)
- [SATr: Slice Attention with Transformer for Universal Lesion Detection](https://mlsanity.com/paper/2203.07373/) (2022)
- [Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction](https://mlsanity.com/paper/2205.07471/) (2022)
- [Rethinking Few-Shot Object Detection on a Multi-Domain Benchmark](https://mlsanity.com/paper/2207.11169/) (2022)
- [Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration](https://mlsanity.com/paper/2312.15701/) (2023)
- [CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans](https://mlsanity.com/paper/2301.12291/) (2023)
- [Asymmetric 3D Context Fusion for Universal Lesion Detection](https://mlsanity.com/paper/2109.08684/) (2021)
- [Unsupervised CT Metal Artifact Reduction by Plugging Diffusion Priors in Dual Domains](https://mlsanity.com/paper/2308.16742/) (2023)
- [DAN-Net: Dual-Domain Adaptive-Scaling Non-local Network for CT Metal Artifact Reduction](https://mlsanity.com/paper/2102.08003/) (2021)
- [Unsupervised Polychromatic Neural Representation for CT Metal Artifact Reduction](https://mlsanity.com/paper/2306.15203/) (2023)
- [U-DuDoNet: Unpaired dual-domain network for CT metal artifact reduction](https://mlsanity.com/paper/2103.04552/) (2021)
- [IDOL-Net: An Interactive Dual-Domain Parallel Network for CT Metal Artifact Reduction](https://mlsanity.com/paper/2104.01405/) (2021)
- [Metal Artifact Reduction in 2D CT Images with Self-supervised Cross-domain Learning](https://mlsanity.com/paper/2109.13483/) (2021)
- [Transformer Lesion Tracker](https://mlsanity.com/paper/2206.06252/) (2022)
- [Orientation-Shared Convolution Representation for CT Metal Artifact Learning](https://mlsanity.com/paper/2212.13166/) (2022)
- [MEPNet: A Model-Driven Equivariant Proximal Network for Joint Sparse-View Reconstruction and Metal Artifact Reduction in CT Images](https://mlsanity.com/paper/2306.14274/) (2023)
- [Conditional Training with Bounding Map for Universal Lesion Detection](https://mlsanity.com/paper/2103.12277/) (2021)
- [A Regularization-Guided Equivariant Approach for Image Restoration](https://mlsanity.com/paper/2505.19799/) (2025)
- [Unsupervised Multi-Parameter Inverse Solving for Reducing Ring Artifacts in 3D X-Ray CBCT](https://mlsanity.com/paper/2412.05853/) (2024)
- [Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric Approach](https://mlsanity.com/paper/2506.03238/) (2025)
- [Disentangled Learning Improves Implicit Neural Representations for Medical Reconstruction](https://mlsanity.com/paper/2605.04234/) (2026)
- [Text Embedded Swin-UMamba for DeepLesion Segmentation](https://mlsanity.com/paper/2508.06453/) (2025)
- [Glance and Focus Reinforcement for Pan-cancer Screening](https://mlsanity.com/paper/2601.19103/) (2026)
- [Attention-based Dynamic Subspace Learners for Medical Image Analysis](https://mlsanity.com/paper/2206.09068/) (2022)

## Abstract (arXiv)

Radiologists in their daily work routinely find and annotate significant abnormalities on a large number of radiology images. Such abnormalities, or lesions, have collected over years and stored in hospitals' picture archiving and communication systems. However, they are basically unsorted and lack semantic annotations like type and location. In this paper, we aim to organize and explore them by learning a deep feature representation for each lesion. A large-scale and comprehensive dataset, DeepLesion, is introduced for this task. DeepLesion contains bounding boxes and size measurements of over 32K lesions. To model their similarity relationship, we leverage multiple supervision information including types, self-supervised location coordinates and sizes. They require little manual annotation effort but describe useful attributes of the lesions. Then, a triplet network is utilized to learn lesion embeddings with a sequential sampling strategy to depict their hierarchical similarity structure. Experiments show promising qualitative and quantitative results on lesion retrieval, clustering, and classification. The learned embeddings can be further employed to build a lesion graph for various clinically useful applications. We propose algorithms for intra-patient lesion matching and missing annotation mining. Experimental results validate their effectiveness.