# Transformers Improve Breast Cancer Diagnosis from Unregistered Multi-View Mammograms

arXiv: https://arxiv.org/abs/2206.10096 · HTML: https://mlsanity.com/paper/2206.10096/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening](https://mlsanity.com/paper/1903.08297/) (2019)
- [TransMed: Transformers Advance Multi-modal Medical Image Classification](https://mlsanity.com/paper/2103.05940/) (2021)
- [Is it Time to Replace CNNs with Transformers for Medical Images?](https://mlsanity.com/paper/2108.09038/) (2021)
- [Multi-view analysis of unregistered medical images using cross-view transformers](https://mlsanity.com/paper/2103.11390/) (2021)
- [MVT: Multi-view Vision Transformer for 3D Object Recognition](https://mlsanity.com/paper/2110.13083/) (2021)

## Built on by (most cited first, 11 of 11)

- [MV-Swin-T: Mammogram Classification with Multi-view Swin Transformer](https://mlsanity.com/paper/2402.16298/) (2024)
- [Recent Progress in Transformer-based Medical Image Analysis](https://mlsanity.com/paper/2208.06643/) (2022)
- [Mammo-CLIP: Leveraging Contrastive Language-Image Pre-training (CLIP) for Enhanced Breast Cancer Diagnosis with Multi-view Mammography](https://mlsanity.com/paper/2404.15946/) (2024)
- [Vision Transformer-based Feature Extraction for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2302.00875/) (2023)
- [Advanced Deep Learning and Large Language Models: Comprehensive Insights for Cancer Detection](https://mlsanity.com/paper/2504.13186/) (2025)
- [A Hybrid CNN-VSSM model for Multi-View, Multi-Task Mammography Analysis: Robust Diagnosis with Attention-Based Fusion](https://mlsanity.com/paper/2507.16955/) (2025)
- [Deep learning in medical image registration: introduction and survey](https://mlsanity.com/paper/2309.00727/) (2023)
- [Evaluating the Effectiveness of 2D and 3D Features for Predicting Tumor Response to Chemotherapy](https://mlsanity.com/paper/2303.16123/) (2023)
- [Random Token Fusion for Multi-View Medical Diagnosis](https://mlsanity.com/paper/2410.15847/) (2024)
- [Integrating AI for Human-Centric Breast Cancer Diagnostics: A Multi-Scale and Multi-View Swin Transformer Framework](https://mlsanity.com/paper/2503.13309/) (2025)
- [Transformer-Based Explainable Deep Learning for Breast Cancer Detection in Mammography: The MammoFormer Framework](https://mlsanity.com/paper/2508.06137/) (2025)

## Abstract (arXiv)

Deep convolutional neural networks (CNNs) have been widely used in various medical imaging tasks. However, due to the intrinsic locality of convolution operation, CNNs generally cannot model long-range dependencies well, which are important for accurately identifying or mapping corresponding breast lesion features computed from unregistered multiple mammograms. This motivates us to leverage the architecture of Multi-view Vision Transformers to capture long-range relationships of multiple mammograms from the same patient in one examination. For this purpose, we employ local Transformer blocks to separately learn patch relationships within four mammograms acquired from two-view (CC/MLO) of two-side (right/left) breasts. The outputs from different views and sides are concatenated and fed into global Transformer blocks, to jointly learn patch relationships between four images representing two different views of the left and right breasts. To evaluate the proposed model, we retrospectively assembled a dataset involving 949 sets of mammograms, which include 470 malignant cases and 479 normal or benign cases. We trained and evaluated the model using a five-fold cross-validation method. Without any arduous preprocessing steps (e.g., optimal window cropping, chest wall or pectoral muscle removal, two-view image registration, etc.), our four-image (two-view-two-side) Transformer-based model achieves case classification performance with an area under ROC curve (AUC = 0.818), which significantly outperforms AUC = 0.784 achieved by the state-of-the-art multi-view CNNs (p = 0.009). It also outperforms two one-view-two-side models that achieve AUC of 0.724 (CC view) and 0.769 (MLO view), respectively. The study demonstrates the potential of using Transformers to develop high-performing computer-aided diagnosis schemes that combine four mammograms.