# An End-to-End Breast Tumour Classification Model Using Context-Based Patch Modelling- A BiLSTM Approach for Image Classification

arXiv: https://arxiv.org/abs/2106.02864 · HTML: https://mlsanity.com/paper/2106.02864/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 5 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [DenseCap: Fully Convolutional Localization Networks for Dense Captioning](https://mlsanity.com/paper/1511.07571/) (2015)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [BACH: Grand Challenge on Breast Cancer Histology Images](https://mlsanity.com/paper/1808.04277/) (2018)
- [Data Augmentation by Pairing Samples for Images Classification](https://mlsanity.com/paper/1801.02929/) (2018)
- [Context-Aware Convolutional Neural Network for Grading of Colorectal Cancer Histology Images](https://mlsanity.com/paper/1907.09478/) (2019)
- [RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification](https://mlsanity.com/paper/2010.06440/) (2020)
- [ReNet: A Recurrent Neural Network Based Alternative to Convolutional Networks](https://mlsanity.com/paper/1505.00393/) (2015)
- [Multi-Label Image Classification with Regional Latent Semantic Dependencies](https://mlsanity.com/paper/1612.01082/) (2016)
- [Grid Long Short-Term Memory](https://mlsanity.com/paper/1507.01526/) (2015)
- [Learning Where to See: A Novel Attention Model for Automated Immunohistochemical Scoring](https://mlsanity.com/paper/1903.10762/) (2019)
- [Improving High Resolution Histology Image Classification with Deep Spatial Fusion Network](https://mlsanity.com/paper/1807.10552/) (2018)

## Built on by (most cited first, 3 of 3)

- [Dual Attention Model with Reinforcement Learning for Classification of Histology Whole-Slide Images](https://mlsanity.com/paper/2302.09682/) (2023)
- [Weighted multi-level deep learning analysis and framework for processing breast cancer WSIs](https://mlsanity.com/paper/2106.14708/) (2021)
- [DPSeq: A Novel and Efficient Digital Pathology Classifier for Predicting Cancer Biomarkers using Sequencer Architecture](https://mlsanity.com/paper/2305.01968/) (2023)

## Abstract (arXiv)

Researchers working on computational analysis of Whole Slide Images (WSIs) in histopathology have primarily resorted to patch-based modelling due to large resolution of each WSI. The large resolution makes WSIs infeasible to be fed directly into the machine learning models due to computational constraints. However, due to patch-based analysis, most of the current methods fail to exploit the underlying spatial relationship among the patches. In our work, we have tried to integrate this relationship along with feature-based correlation among the extracted patches from the particular tumorous region. For the given task of classification, we have used BiLSTMs to model both forward and backward contextual relationship. RNN based models eliminate the limitation of sequence size by allowing the modelling of variable size images within a deep learning model. We have also incorporated the effect of spatial continuity by exploring different scanning techniques used to sample patches. To establish the efficiency of our approach, we trained and tested our model on two datasets, microscopy images and WSI tumour regions. After comparing with contemporary literature we achieved the better performance with accuracy of 90% for microscopy image dataset. For WSI tumour region dataset, we compared the classification results with deep learning networks such as ResNet, DenseNet, and InceptionV3 using maximum voting technique. We achieved the highest performance accuracy of 84%. We found out that BiLSTMs with CNN features have performed much better in modelling patches into an end-to-end Image classification network. Additionally, the variable dimensions of WSI tumour regions were used for classification without the need for resizing. This suggests that our method is independent of tumour image size and can process large dimensional images without losing the resolution details.