mlsanityData to 7 Oct 2026

CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans

arXiv 2301.12291 ↗First posted 28 Jan 2023eess.IV cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice, while most medical AI systems are built to focus on single organs with a narrow list of a few diseases. This might severely limit AI's clinical adoption. A certain number of AI models need to be assembled non-trivially to match the diagnostic process of a human reading a CT scan. In this paper, we construct a Unified Tumor Transformer (CancerUniT) model to jointly detect tumor existence & location and diagnose tumor characteristics for eight major cancers in CT scans. CancerUniT is a query-based Mask Transformer model with the output of multi-tumor prediction. We decouple the object queries into organ queries, tumor detection queries and tumor diagnosis queries, and further establish hierarchical relationships among the three groups. This clinically-inspired architecture effectively assists inter- and intra-organ representation learning of tumors and facilitates the resolution of these complex, anatomically related multi-organ cancer image reading tasks. CancerUniT is trained end-to-end using a curated large-scale CT images of 10,042 patients including eight major types of cancers and occurring non-cancer tumors (all are pathology-confirmed with 3D tumor masks annotated by radiologists). On the test set of 631 patients, CancerUniT has demonstrated strong performance under a set of clinically relevant evaluation metrics, substantially outperforming both multi-disease methods and an assembly of eight single-organ expert models in tumor detection, segmentation, and diagnosis. This moves one step closer towards a universal high performance cancer screening tool.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 40 of its 82 references that are in the map. First posted on arXiv 28 Jan 2023.

Builds on

40 of 82 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  10. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  11. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  12. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
28 more
  1. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  2. 2021Multiscale Vision Transformerscited by 896
  3. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  4. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 823
  5. 2021The Medical Segmentation Decathloncited by 620
  6. 2019UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentationcited by 583
  7. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 504
  8. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  9. 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
  10. 2019The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challengecited by 188
  11. 2019A deep learning system for differential diagnosis of skin diseasescited by 118
  12. 2020DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasetscited by 94
  13. 2019Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentationcited by 88
  14. 2021TransMix: Attend to Mix for Vision Transformerscited by 79
  15. 2021Modality-aware Mutual Learning for Multi-modal Medical Image Segmentationcited by 47
  16. 2018Abdominal multi-organ segmentation with organ-attention networks and statistical fusioncited by 46
  17. 2017Deep Lesion Graphs in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-scale Lesion Databasecited by 42
  18. 2020Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CTcited by 42
  19. 2019MULAN: Multitask Universal Lesion Analysis Network for Joint Lesion Detection, Tagging, and Segmentationcited by 39
  20. 2018Multi-Scale Coarse-to-Fine Segmentation for Screening Pancreatic Ductal Adenocarcinomacited by 24
  21. 2019ULDor: A Universal Lesion Detector for CT Scans with Pseudo Masks and Hard Negative Example Miningcited by 21
  22. 20203D Graph Anatomy Geometry-Integrated Network for Pancreatic Mass Segmentation, Diagnosis, and Quantitative Patient Managementcited by 19
  23. 2019Holistic and Comprehensive Annotation of Clinically Significant Findings on Diverse CT Images: Learning from Radiology Reports and Label Ontologycited by 18
  24. 2021Weakly-Supervised Universal Lesion Segmentation with Regional Level Set Losscited by 16
  25. 2021Asymmetric 3D Context Fusion for Universal Lesion Detectioncited by 14
  26. 2019Automatic Pulmonary Nodule Detection in CT Scans Using Convolutional Neural Networks Based on Maximum Intensity Projectioncited by 10
  27. 2020Harvesting, Detecting, and Characterizing Liver Lesions from Large-scale Multi-phase CT Data via Deep Dynamic Texture Learningcited by 7
  28. 2019Semi-Supervised Multi-Organ Segmentation through Quality Assurance Supervisioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 6; 2026: 1.

15 papers.

Sort:

By area:

  1. 2024Towards Generalizable Tumor Synthesiscited by 65
  2. 2025RadGPT: Constructing 3D Image-Text Tumor Datasetscited by 34
  3. 2024Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomographycited by 31
  4. 2024Automatic Organ and Pan-cancer Segmentation in Abdomen CT: the FLARE 2023 Challengecited by 20
  5. 2023ZePT: Zero-Shot Pan-Tumor Segmentation via Query-Disentangling and Self-Promptingcited by 15
  6. 2023Segment Together: A Versatile Paradigm for Semi-Supervised Medical Image Segmentationcited by 12
  7. 2023Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Datacited by 9
  8. 2025Scaling Artificial Intelligence for Multi-Tumor Early Detection with More Reports, Fewer Maskscited by 8
  9. 2024Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignmentcited by 7
  10. 2025Mamba Goes HoME: Hierarchical Soft Mixture-of-Experts for 3D Medical Image Segmentationcited by 3
  11. 2024Boosting Medical Image-based Cancer Detection via Text-guided Supervision from Reportscited by 2
  12. 2025Advancing Generalizable Tumor Segmentation with Anomaly-Aware Open-Vocabulary Attention Maps and Frozen Foundation Diffusion Modelscited by 1
3 more
  1. 2026Glance and Focus Reinforcement for Pan-cancer Screeningcited by 1
  2. 2025MAST-Pro: Dynamic Mixture-of-Experts for Adaptive Segmentation of Pan-Tumors with Knowledge-Driven Promptscited by 0
  3. 2025Beyond Size and Growth: Rethinking Lung Cancer Screening with AI Based Nodule Detection and Diagnosiscited by 0