# DSCA: A Dual-Stream Network with Cross-Attention on Whole-Slide Image Pyramids for Cancer Prognosis

arXiv: https://arxiv.org/abs/2206.05782 · HTML: https://mlsanity.com/paper/2206.05782/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 12 Jun 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks](https://mlsanity.com/paper/2107.13048/) (2021)
- [Revisiting Multiple Instance Neural Networks](https://mlsanity.com/paper/1610.02501/) (2016)
- [Feature Pyramid Transformer](https://mlsanity.com/paper/2007.09451/) (2020)
- [Hierarchical Graph Representations in Digital Pathology](https://mlsanity.com/paper/2102.11057/) (2021)
- [HACT-Net: A Hierarchical Cell-to-Tissue Graph Neural Network for Histopathological Image Classification](https://mlsanity.com/paper/2007.00584/) (2020)

## Built on by (most cited first, 10 of 10)

- [AdvMIL: Adversarial Multiple Instance Learning for the Survival Analysis on Whole-Slide Images](https://mlsanity.com/paper/2212.06515/) (2022)
- [Interpretable Vision-Language Survival Analysis with Ordinal Inductive Bias for Computational Pathology](https://mlsanity.com/paper/2409.09369/) (2024)
- [Dual Attention Model with Reinforcement Learning for Classification of Histology Whole-Slide Images](https://mlsanity.com/paper/2302.09682/) (2023)
- [ProtoDiv: Prototype-guided Division of Consistent Pseudo-bags for Whole-slide Image Classification](https://mlsanity.com/paper/2304.06652/) (2023)
- [Multi-scale Efficient Graph-Transformer for Whole Slide Image Classification](https://mlsanity.com/paper/2305.15773/) (2023)
- [cDP-MIL: Robust Multiple Instance Learning via Cascaded Dirichlet Process](https://mlsanity.com/paper/2407.11448/) (2024)
- [Self-Contrastive Weakly Supervised Learning Framework for Prognostic Prediction Using Whole Slide Images](https://mlsanity.com/paper/2405.15264/) (2024)
- [M2EF-NNs: Multimodal Multi-instance Evidence Fusion Neural Networks for Cancer Survival Prediction](https://mlsanity.com/paper/2408.04170/) (2024)
- [Cross-Patient Pseudo Bags Generation and Curriculum Contrastive Learning for Imbalanced Multiclassification of Whole Slide Image](https://mlsanity.com/paper/2411.11262/) (2024)
- [Parameter-Efficient Subspace Decoupling ViT for Mitigating Multi-Task Negative Transfer in Histological Scoring](https://mlsanity.com/paper/2605.29852/) (2026)

## Abstract (arXiv)

The cancer prognosis on gigapixel Whole-Slide Images (WSIs) has always been a challenging task. To further enhance WSI visual representations, existing methods have explored image pyramids, instead of single-resolution images, in WSIs. In spite of this, they still face two major problems: high computational cost and the unnoticed semantical gap in multi-resolution feature fusion. To tackle these problems, this paper proposes to efficiently exploit WSI pyramids from a new perspective, the dual-stream network with cross-attention (DSCA). Our key idea is to utilize two sub-streams to process the WSI patches with two resolutions, where a square pooling is devised in a high-resolution stream to significantly reduce computational costs, and a cross-attention-based method is proposed to properly handle the fusion of dual-stream features. We validate our DSCA on three publicly-available datasets with a total number of 3,101 WSIs from 1,911 patients. Our experiments and ablation studies verify that (i) the proposed DSCA could outperform existing state-of-the-art methods in cancer prognosis, by an average C-Index improvement of around 4.6%; (ii) our DSCA network is more efficient in computation -- it has more learnable parameters (6.31M vs. 860.18K) but less computational costs (2.51G vs. 4.94G), compared to a typical existing multi-resolution network. (iii) the key components of DSCA, dual-stream and cross-attention, indeed contribute to our model's performance, gaining an average C-Index rise of around 2.0% while maintaining a relatively-small computational load. Our DSCA could serve as an alternative and effective tool for WSI-based cancer prognosis.