# Multi-Granularity 3D Kidney Lesion Characterization from CT Volumes

arXiv: https://arxiv.org/abs/2606.04365 · HTML: https://mlsanity.com/paper/2606.04365/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 24 of 24)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?](https://mlsanity.com/paper/1706.00712/) (2017)
- [Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography](https://mlsanity.com/paper/2403.17834/) (2024)
- [M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models](https://mlsanity.com/paper/2404.00578/) (2024)
- [Merlin: A Computed Tomography Vision-Language Foundation Model and Dataset](https://mlsanity.com/paper/2406.06512/) (2024)
- [STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training](https://mlsanity.com/paper/2304.06716/) (2023)
- [VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis](https://mlsanity.com/paper/2402.17300/) (2024)
- [MULAN: Multitask Universal Lesion Analysis Network for Joint Lesion Detection, Tagging, and Segmentation](https://mlsanity.com/paper/1908.04373/) (2019)
- [RadGPT: Constructing 3D Image-Text Tumor Datasets](https://mlsanity.com/paper/2501.04678/) (2025)
- [Improving RetinaNet for CT Lesion Detection with Dense Masks from Weak RECIST Labels](https://mlsanity.com/paper/1906.02283/) (2019)
- [Models Genesis](https://mlsanity.com/paper/2004.07882/) (2020)
- [Automated 3D Segmentation of Kidneys and Tumors in MICCAI KiTS 2023 Challenge](https://mlsanity.com/paper/2310.04110/) (2023)
- [How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?](https://mlsanity.com/paper/2501.11253/) (2025)
- [CT-GRAPH: Hierarchical Graph Attention Network for Anatomy-Guided CT Report Generation](https://mlsanity.com/paper/2508.05375/) (2025)
- [Lesion-Aware Cross-Phase Attention Network for Renal Tumor Subtype Classification on Multi-Phase CT Scans](https://mlsanity.com/paper/2406.16322/) (2024)
- [Exploring the Design Space of 3D MLLMs for CT Report Generation](https://mlsanity.com/paper/2506.21535/) (2025)
- [A Clinically-Grounded Two-Stage Framework for Renal CT Report Generation](https://mlsanity.com/paper/2506.23584/) (2025)

## Abstract (arXiv)

Radiology reports describe kidney lesions by type, size, enhancement, and attenuation, yet existing 3D methods predict only at the patient or organ level. We reformulate kidney CT characterization as a per-lesion set-prediction task: one model emits a variable number of lesions per kidney, each with four clinical attributes. We curated 2,619 CT volumes from 788 patients at one academic medical center, with multi-granularity side- and per-lesion labels, and used KiTS23 (489 cases) for zero-shot external validation. We propose LesionDETR, a DETR-style architecture with size-distance Hungarian matching and a hierarchical loss that aggregates per-slot outputs to side-level objectives. Across four input representations and six encoder initializations, two design choices dominate: a segmentation mask as an input channel, and same-domain abdominal pretraining (SuPreM); generic large-corpus pretraining is no better than random initialization. LesionDETR reaches bilateral side-level abnormality AUC 0.799 ± 0.009 on UF-Health and 0.817 ± 0.072 on KiTS23. A count-conditioned variant reaches per-lesion mAP 0.190 ± 0.083 on cystic lesions; rare solid-lesion AP stays at the noise floor, pointing to targeted data collection, not architecture, as the next bottleneck. The framework yields verified per-lesion predictions for downstream structured report generation.