# Unsupervised Contrastive Learning based Transformer for Lung Nodule Detection

arXiv: https://arxiv.org/abs/2205.00122 · HTML: https://mlsanity.com/paper/2205.00122/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 69 references that are in the map.
- First posted on arXiv 30 Apr 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [SPICE: Semantic Pseudo-labeling for Image Clustering](https://mlsanity.com/paper/2103.09382/) (2021)
- [nnDetection: A Self-configuring Method for Medical Object Detection](https://mlsanity.com/paper/2106.00817/) (2021)
- [Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection](https://mlsanity.com/paper/1811.08661/) (2018)
- [Face Detection with the Faster R-CNN](https://mlsanity.com/paper/1606.03473/) (2016)
- [GATCluster: Self-Supervised Gaussian-Attention Network for Image Clustering](https://mlsanity.com/paper/2002.11863/) (2020)
- [Multi-domain Integrative Swin Transformer network for Sparse-View Tomographic Reconstruction](https://mlsanity.com/paper/2111.14831/) (2021)
- [Progressive and Multi-Path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images](https://mlsanity.com/paper/1706.03702/) (2017)
- [ProCAN: Progressive Growing Channel Attentive Non-Local Network for Lung Nodule Classification](https://mlsanity.com/paper/2010.15417/) (2020)
- [Relational Learning between Multiple Pulmonary Nodules via Deep Set Attention Transformers](https://mlsanity.com/paper/2004.05640/) (2020)
- [X-ray Dissectography Improves Lung Nodule Detection](https://mlsanity.com/paper/2203.13118/) (2022)

## Built on by (most cited first, 11 of 11)

- [Forget Less, Count Better: A Domain-Incremental Self-Distillation Learning Benchmark for Lifelong Crowd Counting](https://mlsanity.com/paper/2205.03307/) (2022)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [Specialty-Oriented Generalist Medical AI for Chest CT Screening](https://mlsanity.com/paper/2304.02649/) (2023)
- [A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images](https://mlsanity.com/paper/2309.02555/) (2023)
- [Improving diagnosis and prognosis of lung cancer using vision transformers: A scoping review](https://mlsanity.com/paper/2309.02783/) (2023)
- [HOME: High-Order Mixed-Moment-based Embedding for Representation Learning](https://mlsanity.com/paper/2207.07743/) (2022)
- [Lung-CADex: Fully automatic Zero-Shot Detection and Classification of Lung Nodules in Thoracic CT Images](https://mlsanity.com/paper/2407.02625/) (2024)
- [Exploring learning environments for label-efficient cancer diagnosis](https://mlsanity.com/paper/2408.07988/) (2024)
- [Development and Validation of a Large Language Model for Generating Fully-Structured Radiology Reports](https://mlsanity.com/paper/2409.18319/) (2024)
- [EMeRALDS: Electronic Medical Record Driven Automated Lung Nodule Detection and Classification in Thoracic CT Images](https://mlsanity.com/paper/2509.11714/) (2025)
- [Foundation Models for Medical Imaging: Status, Challenges, and Directions](https://mlsanity.com/paper/2602.15913/) (2026)

## Abstract (arXiv)

Early detection of lung nodules with computed tomography (CT) is critical for the longer survival of lung cancer patients and better quality of life. Computer-aided detection/diagnosis (CAD) is proven valuable as a second or concurrent reader in this context. However, accurate detection of lung nodules remains a challenge for such CAD systems and even radiologists due to not only the variability in size, location, and appearance of lung nodules but also the complexity of lung structures. This leads to a high false-positive rate with CAD, compromising its clinical efficacy. Motivated by recent computer vision techniques, here we present a self-supervised region-based 3D transformer model to identify lung nodules among a set of candidate regions. Specifically, a 3D vision transformer (ViT) is developed that divides a CT image volume into a sequence of non-overlap cubes, extracts embedding features from each cube with an embedding layer, and analyzes all embedding features with a self-attention mechanism for the prediction. To effectively train the transformer model on a relatively small dataset, the region-based contrastive learning method is used to boost the performance by pre-training the 3D transformer with public CT images. Our experiments show that the proposed method can significantly improve the performance of lung nodule screening in comparison with the commonly used 3D convolutional neural networks.