# DeepGene Transformer: Transformer for the gene expression-based classification of cancer subtypes

arXiv: https://arxiv.org/abs/2108.11833 · HTML: https://mlsanity.com/paper/2108.11833/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 41 references that are in the map.
- First posted on arXiv 26 Aug 2021.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [1D Convolutional Neural Networks and Applications: A Survey](https://mlsanity.com/paper/1905.03554/) (2019)
- [Convolutional neural network models for cancer type prediction based on gene expression](https://mlsanity.com/paper/1906.07794/) (2019)
- [Convolution, attention and structure embedding](https://mlsanity.com/paper/1905.01289/) (2019)

## Built on by (most cited first, 5 of 5)

- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks](https://mlsanity.com/paper/2306.07303/) (2023)
- [A survey on attention mechanisms for medical applications: are we moving towards better algorithms?](https://mlsanity.com/paper/2204.12406/) (2022)
- [Interpretable Medical Imagery Diagnosis with Self-Attentive Transformers: A Review of Explainable AI for Health Care](https://mlsanity.com/paper/2309.00252/) (2023)
- [Federated Learning in Genetics: Extended Analysis of Accuracy, Performance and Privacy Trade-offs](https://mlsanity.com/paper/2402.14527/) (2024)

## Abstract (arXiv)

Cancer and its subtypes constitute approximately 30% of all causes of death globally and display a wide range of heterogeneity in terms of clinical and molecular responses to therapy. Molecular subtyping has enabled the use of precision medicine to overcome these challenges and provide significant biological insights to predict prognosis and improve clinical decision-making. Over the past decade, conventional machine learning (ML) and deep learning (DL) algorithms have been widely espoused for the classification of cancer subtypes from gene expression datasets. However, these methods are potentially biased toward the identification of cancer biomarkers. Hence, an end-to-end deep learning approach, DeepGene Transformer, is proposed which addresses the complexity of high-dimensional gene expression with a multi-head self-attention module by identifying relevant biomarkers across multiple cancer subtypes without requiring feature selection as a pre-requisite for the current classification algorithms. Comparative analysis reveals that the proposed DeepGene Transformer outperformed the commonly used traditional and state-of-the-art classification algorithms and can be considered an efficient approach for classifying cancer and its subtypes, indicating that any improvement in deep learning models in computational biologists can be reflected well in this domain as well.