# A Unified 2D Framework for DeepLesion Detection, Segmentation and Short Report Generation

arXiv: https://arxiv.org/abs/2608.02805 · HTML: https://mlsanity.com/paper/2608.02805/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [Generating Radiology Reports via Memory-driven Transformer](https://mlsanity.com/paper/2010.16056/) (2020)
- [YOLOv12: Attention-Centric Real-Time Object Detectors](https://mlsanity.com/paper/2502.12524/) (2025)
- [Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining](https://mlsanity.com/paper/2402.03302/) (2024)
- [Interactive and Explainable Region-guided Radiology Report Generation](https://mlsanity.com/paper/2304.08295/) (2023)
- [Describe Anything: Detailed Localized Image and Video Captioning](https://mlsanity.com/paper/2504.16072/) (2025)
- [MedGemma 1.5 Technical Report](https://mlsanity.com/paper/2604.05081/) (2026)
- [MULAN: Multitask Universal Lesion Analysis Network for Joint Lesion Detection, Tagging, and Segmentation](https://mlsanity.com/paper/1908.04373/) (2019)
- [Accurate Weakly-Supervised Deep Lesion Segmentation using Large-Scale Clinical Annotations: Slice-Propagated 3D Mask Generation from 2D RECIST](https://mlsanity.com/paper/1807.01172/) (2018)
- [Weakly-Supervised Universal Lesion Segmentation with Regional Level Set Loss](https://mlsanity.com/paper/2105.01218/) (2021)
- [Lesion Harvester: Iteratively Mining Unlabeled Lesions and Hard-Negative Examples at Scale](https://mlsanity.com/paper/2001.07776/) (2020)
- [R2Gen-Mamba: A Selective State Space Model for Radiology Report Generation](https://mlsanity.com/paper/2410.18135/) (2024)
- [Enhancing Radiology Report Generation and Visual Grounding using Reinforcement Learning](https://mlsanity.com/paper/2512.10691/) (2025)
- [YOLO-Master: MOE-Accelerated with Specialized Transformers for Enhanced Real-time Detection](https://mlsanity.com/paper/2512.23273/) (2025)
- [Text Embedded Swin-UMamba for DeepLesion Segmentation](https://mlsanity.com/paper/2508.06453/) (2025)

## Abstract (arXiv)

In previous work, we integrated large language models (LLMs) into the lesion segmentation model based on the ULS23 DeepLesion dataset, using short-form findings from the reports. In this study, we developed a unified 2D lesion analysis framework that integrates LLM-based reasoning, lesion bounding box detection, segmentation, and radiology report generation from the original DeepLesion dataset. In the testing phase, we achieved relatively high lesion bounding box detection accuracy with mAP50 of 70.1%, mAP50-95 of 46.4%; Lesion segmentation performance with a Dice score of 62.6%; short report generation accuracy with BLEU_1 score of 64.3%, BLEU_4 score of 49.6%, METEOR of 34.7%, and ROUGE_L of 60.1%. In this work, we address the challenging issue of segmentation in the original DeepLesion dataset and achieve a 28.5% Dice score improvement over the nnUNet lesion segmentation model. We also integrated spatial and anatomical context into the DeepLesion short report generation. We released the implementation, dataset, and models on Github. https://github.com/ruida/2D_DeepLesion_Foundation