# DeepProphet2 -- A Deep Learning Gene Recommendation Engine

arXiv: https://arxiv.org/abs/2208.01918 · HTML: https://mlsanity.com/paper/2208.01918/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 56 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Self-Alignment Pretraining for Biomedical Entity Representations](https://mlsanity.com/paper/2010.11784/) (2020)
- [Drug discovery with explainable artificial intelligence](https://mlsanity.com/paper/2007.00523/) (2020)
- [The Geometry of Culture: Analyzing Meaning through Word Embeddings](https://mlsanity.com/paper/1803.09288/) (2018)
- [Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS](https://mlsanity.com/paper/1912.06059/) (2019)
- [Unsupervised Pretraining for Sequence to Sequence Learning](https://mlsanity.com/paper/1611.02683/) (2016)
- [Knowledge-Rich Self-Supervision for Biomedical Entity Linking](https://mlsanity.com/paper/2112.07887/) (2021)
- [Understanding Autoencoders with Information Theoretic Concepts](https://mlsanity.com/paper/1804.00057/) (2018)
- [Artificial Intelligence in Drug Discovery: Applications and Techniques](https://mlsanity.com/paper/2106.05386/) (2021)
- [Principled approach to the selection of the embedding dimension of networks](https://mlsanity.com/paper/2004.09928/) (2020)
- [SECNLP: A Survey of Embeddings in Clinical Natural Language Processing](https://mlsanity.com/paper/1903.01039/) (2019)
- [Efficient Representation of Low-Dimensional Manifolds using Deep Networks](https://mlsanity.com/paper/1602.04723/) (2016)
- [Empirical comparison between autoencoders and traditional dimensionality reduction methods](https://mlsanity.com/paper/2103.04874/) (2021)
- [Categorical Representation Learning: Morphism is All You Need](https://mlsanity.com/paper/2103.14770/) (2021)
- [Deep learning models are not robust against noise in clinical text](https://mlsanity.com/paper/2108.12242/) (2021)
- [Theoretical foundations and limits of word embeddings: what types of meaning can they capture?](https://mlsanity.com/paper/2107.10413/) (2021)
- [Understand Functionality and Dimensionality of Vector Embeddings: the Distributional Hypothesis, the Pairwise Inner Product Loss and Its Bias-Variance Trade-off](https://mlsanity.com/paper/1803.00502/) (2018)
- [A DIKW Paradigm to Cognitive Engineering](https://mlsanity.com/paper/1702.07168/) (2017)
- [Generating Summaries for Scientific Paper Review](https://mlsanity.com/paper/2109.14059/) (2021)

## Abstract (arXiv)

New powerful tools for tackling life science problems have been created by recent advances in machine learning. The purpose of the paper is to discuss the potential advantages of gene recommendation performed by artificial intelligence (AI). Indeed, gene recommendation engines try to solve this problem: if the user is interested in a set of genes, which other genes are likely to be related to the starting set and should be investigated? This task was solved with a custom deep learning recommendation engine, DeepProphet2 (DP2), which is freely available to researchers worldwide via https://www.generecommender.com?utm_source=DeepProphet2_paper&utm_medium=pdf. Hereafter, insights behind the algorithm and its practical applications are illustrated. The gene recommendation problem can be addressed by mapping the genes to a metric space where a distance can be defined to represent the real semantic distance between them. To achieve this objective a transformer-based model has been trained on a well-curated freely available paper corpus, PubMed. The paper describes multiple optimization procedures that were employed to obtain the best bias-variance trade-off, focusing on embedding size and network depth. In this context, the model's ability to discover sets of genes implicated in diseases and pathways was assessed through cross-validation. A simple assumption guided the procedure: the network had no direct knowledge of pathways and diseases but learned genes' similarities and the interactions among them. Moreover, to further investigate the space where the neural network represents genes, the dimensionality of the embedding was reduced, and the results were projected onto a human-comprehensible space. In conclusion, a set of use cases illustrates the algorithm's potential applications in a real word setting.