# Auto-weighting for Breast Cancer Classification in Multimodal Ultrasound

arXiv: https://arxiv.org/abs/2008.03435 · HTML: https://mlsanity.com/paper/2008.03435/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2020.

## Built on by (most cited first, 9 of 9)

- [Deep reinforcement learning in medical imaging: A literature review](https://mlsanity.com/paper/2103.05115/) (2021)
- [Reinforcement Learning in Medical Image Analysis: Concepts, Applications, Challenges, and Future Directions](https://mlsanity.com/paper/2206.14302/) (2022)
- [AWSnet: An Auto-weighted Supervision Attention Network for Myocardial Scar and Edema Segmentation in Multi-sequence Cardiac Magnetic Resonance Images](https://mlsanity.com/paper/2201.05344/) (2022)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Image quality assessment by overlapping task-specific and task-agnostic measures: application to prostate multiparametric MR images for cancer segmentation](https://mlsanity.com/paper/2202.09798/) (2022)
- [RLogist: Fast Observation Strategy on Whole-slide Images with Deep Reinforcement Learning](https://mlsanity.com/paper/2212.01737/) (2022)
- [Learning image quality assessment by reinforcing task amenable data selection](https://mlsanity.com/paper/2102.07615/) (2021)
- [Thyroid ultrasound diagnosis improvement via multi-view self-supervised learning and two-stage pre-training](https://mlsanity.com/paper/2402.11497/) (2024)
- [Hierarchical Corpus-View-Category Refinement for Carotid Plaque Risk Grading in Ultrasound](https://mlsanity.com/paper/2506.23108/) (2025)

## Abstract (arXiv)

Breast cancer is the most common invasive cancer in women. Besides the primary B-mode ultrasound screening, sonographers have explored the inclusion of Doppler, strain and shear-wave elasticity imaging to advance the diagnosis. However, recognizing useful patterns in all types of images and weighing up the significance of each modality can elude less-experienced clinicians. In this paper, we explore, for the first time, an automatic way to combine the four types of ultrasonography to discriminate between benign and malignant breast nodules. A novel multimodal network is proposed, along with promising learnability and simplicity to improve classification accuracy. The key is using a weight-sharing strategy to encourage interactions between modalities and adopting an additional cross-modalities objective to integrate global information. In contrast to hardcoding the weights of each modality in the model, we embed it in a Reinforcement Learning framework to learn this weighting in an end-to-end manner. Thus the model is trained to seek the optimal multimodal combination without handcrafted heuristics. The proposed framework is evaluated on a dataset contains 1616 set of multimodal images. Results showed that the model scored a high classification accuracy of 95.4%, which indicates the efficiency of the proposed method.