# A generative recommender system with GMM prior for cancer drug generation and sensitivity prediction

arXiv: https://arxiv.org/abs/2206.03555 · HTML: https://mlsanity.com/paper/2206.03555/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 80 references that are in the map.
- First posted on arXiv 7 Jun 2022.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [VAE with a VampPrior](https://mlsanity.com/paper/1705.07120/) (2017)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [Hierarchical Generation of Molecular Graphs using Structural Motifs](https://mlsanity.com/paper/2002.03230/) (2020)
- [Deep Learning based Recommender System: A Survey and New Perspectives](https://mlsanity.com/paper/1707.07435/) (2017)
- [Generative Adversarial User Model for Reinforcement Learning Based Recommendation System](https://mlsanity.com/paper/1812.10613/) (2018)
- [Stick-Breaking Variational Autoencoders](https://mlsanity.com/paper/1605.06197/) (2016)
- [Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation](https://mlsanity.com/paper/1911.03845/) (2019)
- [Multi-Facet Clustering Variational Autoencoders](https://mlsanity.com/paper/2106.05241/) (2021)
- [Sampling-Decomposable Generative Adversarial Recommender](https://mlsanity.com/paper/2011.00956/) (2020)
- [Interpretable Embeddings From Molecular Simulations Using Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1912.12175/) (2019)
- [Adaptive machine learning for protein engineering](https://mlsanity.com/paper/2106.05466/) (2021)
- [Disentangling to Cluster: Gaussian Mixture Variational Ladder Autoencoders](https://mlsanity.com/paper/1909.11501/) (2019)
- [Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding](https://mlsanity.com/paper/1905.11741/) (2019)
- [SeGMA: Semi-Supervised Gaussian Mixture Auto-Encoder](https://mlsanity.com/paper/1906.09333/) (2019)
- [Variational Autoencoder for Anti-Cancer Drug Response Prediction](https://mlsanity.com/paper/2008.09763/) (2020)
- [A molecular generative model with genetic algorithm and tree search for cancer samples](https://mlsanity.com/paper/2112.08959/) (2021)

## Abstract (arXiv)

Recent emergence of high-throughput drug screening assays sparkled an intensive development of machine learning methods, including models for prediction of sensitivity of cancer cell lines to anti-cancer drugs, as well as methods for generation of potential drug candidates. However, a concept of generation of compounds with specific properties and simultaneous modeling of their efficacy against cancer cell lines has not been comprehensively explored. To address this need, we present VADEERS, a Variational Autoencoder-based Drug Efficacy Estimation Recommender System. The generation of compounds is performed by a novel variational autoencoder with a semi-supervised Gaussian Mixture Model (GMM) prior. The prior defines a clustering in the latent space, where the clusters are associated with specific drug properties. In addition, VADEERS is equipped with a cell line autoencoder and a sensitivity prediction network. The model combines data for SMILES string representations of anti-cancer drugs, their inhibition profiles against a panel of protein kinases, cell lines biological features and measurements of the sensitivity of the cell lines to the drugs. The evaluated variants of VADEERS achieve a high r=0.87 Pearson correlation between true and predicted drug sensitivity estimates. We train the GMM prior in such a way that the clusters in the latent space correspond to a pre-computed clustering of the drugs by their inhibitory profiles. We show that the learned latent representations and new generated data points accurately reflect the given clustering. In summary, VADEERS offers a comprehensive model of drugs and cell lines properties and relationships between them, as well as a guided generation of novel compounds.