# Multimodal Breast Lesion Classification Using Cross-Attention Deep Networks

arXiv: https://arxiv.org/abs/2108.09591 · HTML: https://mlsanity.com/paper/2108.09591/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 13 references that are in the map.
- First posted on arXiv 21 Aug 2021.

## Builds on (most cited first, 4 of 4)

- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [End-to-end Training for Whole Image Breast Cancer Diagnosis using An All Convolutional Design](https://mlsanity.com/paper/1711.05775/) (2017)
- [Detecting and classifying lesions in mammograms with Deep Learning](https://mlsanity.com/paper/1707.08401/) (2017)
- [Multimodal Categorization of Crisis Events in Social Media](https://mlsanity.com/paper/2004.04917/) (2020)

## Built on by (most cited first, 1 of 1)

- [Cross-Attention Multimodal Fusion for Breast Cancer Diagnosis: Integrating Mammography and Clinical Data with Explainability](https://mlsanity.com/paper/2508.16000/) (2025)

## Abstract (arXiv)

Accurate breast lesion risk estimation can significantly reduce unnecessary biopsies and help doctors decide optimal treatment plans. Most existing computer-aided systems rely solely on mammogram features to classify breast lesions. While this approach is convenient, it does not fully exploit useful information in clinical reports to achieve the optimal performance. Would clinical features significantly improve breast lesion classification compared to using mammograms alone? How to handle missing clinical information caused by variation in medical practice? What is the best way to combine mammograms and clinical features? There is a compelling need for a systematic study to address these fundamental questions. This paper investigates several multimodal deep networks based on feature concatenation, cross-attention, and co-attention to combine mammograms and categorical clinical variables. We show that the proposed architectures significantly increase the lesion classification performance (average area under ROC curves from 0.89 to 0.94). We also evaluate the model when clinical variables are missing.