# Multi-task Joint Strategies of Self-supervised Representation Learning on Biomedical Networks for Drug Discovery

arXiv: https://arxiv.org/abs/2201.04437 · HTML: https://mlsanity.com/paper/2201.04437/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 102 references that are in the map.
- First posted on arXiv 12 Jan 2022.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Modeling polypharmacy side effects with graph convolutional networks](https://mlsanity.com/paper/1802.00543/) (2018)
- [Drug discovery with explainable artificial intelligence](https://mlsanity.com/paper/2007.00523/) (2020)
- [A Survey of Heterogeneous Information Network Analysis](https://mlsanity.com/paper/1511.04854/) (2015)
- [Self-supervised Learning on Graphs: Deep Insights and New Direction](https://mlsanity.com/paper/2006.10141/) (2020)
- [Adversarial Multi-task Learning for Text Classification](https://mlsanity.com/paper/1704.05742/) (2017)
- [Self Supervision to Distillation for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2109.04075/) (2021)
- [Contrastive Self-supervised Learning for Graph Classification](https://mlsanity.com/paper/2009.05923/) (2020)
- [Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations](https://mlsanity.com/paper/1906.05017/) (2019)
- [Multi-view Graph Contrastive Representation Learning for Drug-Drug Interaction Prediction](https://mlsanity.com/paper/2010.11711/) (2020)
- [Network Medicine Framework for Identifying Drug Repurposing Opportunities for COVID-19](https://mlsanity.com/paper/2004.07229/) (2020)
- [Self-Supervised Graph Representation Learning via Global Context Prediction](https://mlsanity.com/paper/2003.01604/) (2020)
- [A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and Normalization](https://mlsanity.com/paper/1812.06081/) (2018)
- [Drug Similarity Integration Through Attentive Multi-view Graph Auto-Encoders](https://mlsanity.com/paper/1804.10850/) (2018)
- [Optimizing Area Under the Curve Measures via Matrix Factorization for Predicting Drug-Target Interaction with Multiple Similarities](https://mlsanity.com/paper/2105.01545/) (2021)

## Built on by (most cited first, 13 of 13)

- [Rethinking Data Protection in the (Generative) Artificial Intelligence Era](https://mlsanity.com/paper/2507.03034/) (2025)
- [DrugLLM: Open Large Language Model for Few-shot Molecule Generation](https://mlsanity.com/paper/2405.06690/) (2024)
- [Graph Neural Networks in Modern AI-aided Drug Discovery](https://mlsanity.com/paper/2506.06915/) (2025)
- [Knowledge-augmented Graph Machine Learning for Drug Discovery: A Survey](https://mlsanity.com/paper/2302.08261/) (2023)
- [Towards Unified AI Drug Discovery with Multiple Knowledge Modalities](https://mlsanity.com/paper/2305.01523/) (2023)
- [Integrating Chemical Language and Molecular Graph in Multimodal Fused Deep Learning for Drug Property Prediction](https://mlsanity.com/paper/2312.17495/) (2023)
- [MolCAP: Molecular Chemical reActivity pretraining and prompted-finetuning enhanced molecular representation learning](https://mlsanity.com/paper/2306.09187/) (2023)
- [M3H: Multimodal Multitask Machine Learning for Healthcare](https://mlsanity.com/paper/2404.18975/) (2024)
- [PerCNet: Periodic Complete Representation for Crystal Graphs](https://mlsanity.com/paper/2312.14936/) (2023)
- [CADGL: Context-Aware Deep Graph Learning for Predicting Drug-Drug Interactions](https://mlsanity.com/paper/2403.17210/) (2024)
- [Graph Neural Networks for Quantifying Compatibility Mechanisms in Traditional Chinese Medicine](https://mlsanity.com/paper/2411.11474/) (2024)
- [M3-20M: A Large-Scale Multi-Modal Molecule Dataset for AI-driven Drug Design and Discovery](https://mlsanity.com/paper/2412.06847/) (2024)
- [PhenoGnet: A Graph-Based Contrastive Learning Framework for Disease Similarity Prediction](https://mlsanity.com/paper/2509.14037/) (2025)

## Abstract (arXiv)

Self-supervised representation learning (SSL) on biomedical networks provides new opportunities for drug discovery. However, how to effectively combine multiple SSL models is still challenging and has been rarely explored. Therefore, we propose multi-task joint strategies of self-supervised representation learning on biomedical networks for drug discovery, named MSSL2drug. We design six basic SSL tasks inspired by various modality features including structures, semantics, and attributes in heterogeneous biomedical networks. Importantly, fifteen combinations of multiple tasks are evaluated by a graph attention-based multi-task adversarial learning framework in two drug discovery scenarios. The results suggest two important findings. (1) Combinations of multimodal tasks achieve the best performance compared to other multi-task joint models. (2) The local-global combination models yield higher performance than random two-task combinations when there are the same size of modalities. Therefore, we conjecture that the multimodal and local-global combination strategies can be treated as the guideline of multi-task SSL for drug discovery.