# Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network

arXiv: https://arxiv.org/abs/2308.02101 · HTML: https://mlsanity.com/paper/2308.02101/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 4 Aug 2023.

## Builds on (most cited first, 12 of 12)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [BUSIS: A Benchmark for Breast Ultrasound Image Segmentation](https://mlsanity.com/paper/1801.03182/) (2018)
- [Vision Transformer for Classification of Breast Ultrasound Images](https://mlsanity.com/paper/2110.14731/) (2021)
- [HoVer-Trans: Anatomy-aware HoVer-Transformer for ROI-free Breast Cancer Diagnosis in Ultrasound Images](https://mlsanity.com/paper/2205.08390/) (2022)
- [ESTAN: Enhanced Small Tumor-Aware Network for Breast Ultrasound Image Segmentation](https://mlsanity.com/paper/2009.12894/) (2020)
- [Stan: Small tumor-aware network for breast ultrasound image segmentation](https://mlsanity.com/paper/2002.01034/) (2020)
- [Evaluation of Complexity Measures for Deep Learning Generalization in Medical Image Analysis](https://mlsanity.com/paper/2103.03328/) (2021)
- [EMT-NET: Efficient multitask network for computer-aided diagnosis of breast cancer](https://mlsanity.com/paper/2201.04795/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Novel Channel Boosted Residual CNN-Transformer with Regional-Boundary Learning for Breast Cancer Detection](https://mlsanity.com/paper/2503.15008/) (2025)
- [XBusNet: Text-Guided Breast Ultrasound Segmentation via Multimodal Vision-Language Learning](https://mlsanity.com/paper/2509.07213/) (2025)
- [BTMuda: A Bi-level Multi-source unsupervised domain adaptation framework for breast cancer diagnosis](https://mlsanity.com/paper/2408.17054/) (2024)
- [BUSTR: Descriptor-Aware Vision-Language Learning for Breast Ultrasound Report Generation](https://mlsanity.com/paper/2511.20956/) (2025)

## Abstract (arXiv)

Capturing global contextual information plays a critical role in breast ultrasound (BUS) image classification. Although convolutional neural networks (CNNs) have demonstrated reliable performance in tumor classification, they have inherent limitations for modeling global and long-range dependencies due to the localized nature of convolution operations. Vision Transformers have an improved capability of capturing global contextual information but may distort the local image patterns due to the tokenization operations. In this study, we proposed a hybrid multitask deep neural network called Hybrid-MT-ESTAN, designed to perform BUS tumor classification and segmentation using a hybrid architecture composed of CNNs and Swin Transformer components. The proposed approach was compared to nine BUS classification methods and evaluated using seven quantitative metrics on a dataset of 3,320 BUS images. The results indicate that Hybrid-MT-ESTAN achieved the highest accuracy, sensitivity, and F1 score of 82.7%, 86.4%, and 86.0%, respectively.