# HER2 and FISH Status Prediction in Breast Biopsy H&E-Stained Images Using Deep Learning

arXiv: https://arxiv.org/abs/2408.13818 · HTML: https://mlsanity.com/paper/2408.13818/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 24 references that are in the map.
- First posted on arXiv 25 Aug 2024.

## Builds on (most cited first, 4 of 4)

- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [EGFR Mutation Prediction of Lung Biopsy Images using Deep Learning](https://mlsanity.com/paper/2208.12506/) (2022)

## Built on by (most cited first, 5 of 5)

- [Semantic Segmentation Based Quality Control of Histopathology Whole Slide Images](https://mlsanity.com/paper/2410.03289/) (2024)
- [A Cytology Dataset for Early Detection of Oral Squamous Cell Carcinoma](https://mlsanity.com/paper/2506.09661/) (2025)
- [Scalable Whole Slide Image Representation Using K-Mean Clustering and Fisher Vector Aggregation](https://mlsanity.com/paper/2501.12085/) (2025)
- [Predicting Genetic Mutations from Single-Cell Bone Marrow Images in Acute Myeloid Leukemia Using Noise-Robust Deep Learning Models](https://mlsanity.com/paper/2506.12798/) (2025)
- [Feature Learning with Multi-Stage Vision Transformers on Inter-Modality HER2 Status Scoring and Tumor Classification on Whole Slides](https://mlsanity.com/paper/2512.22335/) (2025)

## Abstract (arXiv)

The current standard for detecting human epidermal growth factor receptor 2 (HER2) status in breast cancer patients relies on HER2 amplification, identified through fluorescence in situ hybridization (FISH) or immunohistochemistry (IHC). However, hematoxylin and eosin (H&E) tumor stains are more widely available, and accurately predicting HER2 status using H&E could reduce costs and expedite treatment selection. Deep Learning algorithms for H&E have shown effectiveness in predicting various cancer features and clinical outcomes, including moderate success in HER2 status prediction. In this work, we employed a customized weak supervision classification technique combined with MoCo-v2 contrastive learning to predict HER2 status. We trained our pipeline on 182 publicly available H&E Whole Slide Images (WSIs) from The Cancer Genome Atlas (TCGA), for which annotations by the pathology team at Yale School of Medicine are publicly available. Our pipeline achieved an Area Under the Curve (AUC) of 0.85 across four different test folds. Additionally, we tested our model on 44 H&E slides from the TCGA-BRCA dataset, which had an HER2 score of 2+ and included corresponding HER2 status and FISH test results. These cases are considered equivocal for IHC, requiring an expensive FISH test on their IHC slides for disambiguation. Our pipeline demonstrated an AUC of 0.81 on these challenging H&E slides. Reducing the need for FISH test can have significant implications in cancer treatment equity for underserved populations.