# An efficient deep neural network to find small objects in large 3D images

arXiv: https://arxiv.org/abs/2210.08645 · HTML: https://mlsanity.com/paper/2210.08645/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 50 references that are in the map.
- First posted on arXiv 16 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Evaluating Weakly Supervised Object Localization Methods Right](https://mlsanity.com/paper/2001.07437/) (2020)
- [Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening](https://mlsanity.com/paper/1903.08297/) (2019)
- [An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization](https://mlsanity.com/paper/2002.07613/) (2020)
- [Robust breast cancer detection in mammography and digital breast tomosynthesis using annotation-efficient deep learning approach](https://mlsanity.com/paper/1912.11027/) (2019)
- [High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks](https://mlsanity.com/paper/1703.07047/) (2017)
- [Detection of masses and architectural distortions in digital breast tomosynthesis: a publicly available dataset of 5,060 patients and a deep learning model](https://mlsanity.com/paper/2011.07995/) (2020)
- [Adaptation of a deep learning malignancy model from full-field digital mammography to digital breast tomosynthesis](https://mlsanity.com/paper/2001.08381/) (2020)
- [Joint 2D-3D Breast Cancer Classification](https://mlsanity.com/paper/2002.12392/) (2020)
- [Data-parallel distributed training of very large models beyond GPU capacity](https://mlsanity.com/paper/1811.12174/) (2018)

## Abstract (arXiv)

3D imaging enables accurate diagnosis by providing spatial information about organ anatomy. However, using 3D images to train AI models is computationally challenging because they consist of 10x or 100x more pixels than their 2D counterparts. To be trained with high-resolution 3D images, convolutional neural networks resort to downsampling them or projecting them to 2D. We propose an effective alternative, a neural network that enables efficient classification of full-resolution 3D medical images. Compared to off-the-shelf convolutional neural networks, our network, 3D Globally-Aware Multiple Instance Classifier (3D-GMIC), uses 77.98%-90.05% less GPU memory and 91.23%-96.02% less computation. While it is trained only with image-level labels, without segmentation labels, it explains its predictions by providing pixel-level saliency maps. On a dataset collected at NYU Langone Health, including 85,526 patients with full-field 2D mammography (FFDM), synthetic 2D mammography, and 3D mammography, 3D-GMIC achieves an AUC of 0.831 (95% CI: 0.769-0.887) in classifying breasts with malignant findings using 3D mammography. This is comparable to the performance of GMIC on FFDM (0.816, 95% CI: 0.737-0.878) and synthetic 2D (0.826, 95% CI: 0.754-0.884), which demonstrates that 3D-GMIC successfully classified large 3D images despite focusing computation on a smaller percentage of its input compared to GMIC. Therefore, 3D-GMIC identifies and utilizes extremely small regions of interest from 3D images consisting of hundreds of millions of pixels, dramatically reducing associated computational challenges. 3D-GMIC generalizes well to BCS-DBT, an external dataset from Duke University Hospital, achieving an AUC of 0.848 (95% CI: 0.798-0.896).