# HoVer-Trans: Anatomy-aware HoVer-Transformer for ROI-free Breast Cancer Diagnosis in Ultrasound Images

arXiv: https://arxiv.org/abs/2205.08390 · HTML: https://mlsanity.com/paper/2205.08390/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 17 May 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data](https://mlsanity.com/paper/2002.03366/) (2020)
- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [COVID-VIT: Classification of COVID-19 from CT chest images based on vision transformer models](https://mlsanity.com/paper/2107.01682/) (2021)
- [Deep Learning Based Computer-Aided Systems for Breast Cancer Imaging : A Critical Review](https://mlsanity.com/paper/2010.00961/) (2020)
- [BI-RADS-Net: An Explainable Multitask Learning Approach for Cancer Diagnosis in Breast Ultrasound Images](https://mlsanity.com/paper/2110.04069/) (2021)

## Built on by (most cited first, 15 of 15)

- [U2-BENCH: Benchmarking Large Vision-Language Models on Ultrasound Understanding](https://mlsanity.com/paper/2505.17779/) (2025)
- [Accurate Airway Tree Segmentation in CT Scans via Anatomy-aware Multi-class Segmentation and Topology-guided Iterative Learning](https://mlsanity.com/paper/2306.09116/) (2023)
- [Revisiting Computer-Aided Tuberculosis Diagnosis](https://mlsanity.com/paper/2307.02848/) (2023)
- [Attention-Map Augmentation for Hypercomplex Breast Cancer Classification](https://mlsanity.com/paper/2310.07633/) (2023)
- [Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network](https://mlsanity.com/paper/2308.02101/) (2023)
- [Do Sharpness-based Optimizers Improve Generalization in Medical Image Analysis?](https://mlsanity.com/paper/2408.04065/) (2024)
- [Breast Lesion Diagnosis Using Static Images and Dynamic Video](https://mlsanity.com/paper/2308.09980/) (2023)
- [UMind-VL: A Generalist Ultrasound Vision-Language Model for Unified Grounded Perception and Comprehensive Interpretation](https://mlsanity.com/paper/2511.22256/) (2025)
- [GCSAM: Gradient Centralized Sharpness Aware Minimization](https://mlsanity.com/paper/2501.11584/) (2025)
- [ViKL: A Mammography Interpretation Framework via Multimodal Aggregation of Visual-knowledge-linguistic Features](https://mlsanity.com/paper/2409.15744/) (2024)
- [Experience-Guided Self-Adaptive Cascaded Agents for Breast Cancer Screening and Diagnosis with Reduced Biopsy Referrals](https://mlsanity.com/paper/2602.23899/) (2026)
- [HFS-TriNet: A Three-Branch Collaborative Feature Learning Network for Prostate Cancer Classification from TRUS Videos](https://mlsanity.com/paper/2604.22388/) (2026)
- [Analyzing Image Encoder Choices and Graph Homophily in GCN Frameworks for Breast Ultrasound Classification](https://mlsanity.com/paper/2607.12054/) (2026)
- [Boot-and-Feedback Framework for Generalist-Expert Model Collaboration in Breast Ultrasound Diagnosis](https://mlsanity.com/paper/2608.23974/) (2026)
- [A Concept-informed Interpretable Artificial Intelligence Framework for Breast Ultrasound Diagnosis](https://mlsanity.com/paper/2610.16128/) (2026)

## Abstract (arXiv)

Ultrasonography is an important routine examination for breast cancer diagnosis, due to its non-invasive, radiation-free and low-cost properties. However, the diagnostic accuracy of breast cancer is still limited due to its inherent limitations. It would be a tremendous success if we can precisely diagnose breast cancer by breast ultrasound images (BUS). Many learning-based computer-aided diagnostic methods have been proposed to achieve breast cancer diagnosis/lesion classification. However, most of them require a pre-define ROI and then classify the lesion inside the ROI. Conventional classification backbones, such as VGG16 and ResNet50, can achieve promising classification results with no ROI requirement. But these models lack interpretability, thus restricting their use in clinical practice. In this study, we propose a novel ROI-free model for breast cancer diagnosis in ultrasound images with interpretable feature representations. We leverage the anatomical prior knowledge that malignant and benign tumors have different spatial relationships between different tissue layers, and propose a HoVer-Transformer to formulate this prior knowledge. The proposed HoVer-Trans block extracts the inter- and intra-layer spatial information horizontally and vertically. We conduct and release an open dataset GDPH&SYSUCC for breast cancer diagnosis in BUS. The proposed model is evaluated in three datasets by comparing with four CNN-based models and two vision transformer models via five-fold cross validation. It achieves state-of-the-art classification performance with the best model interpretability. In the meanwhile, our proposed model outperforms two senior sonographers on the breast cancer diagnosis when only one BUS image is given.