# Molecular Joint Representation Learning via Multi-modal Information

arXiv: https://arxiv.org/abs/2211.14042 · HTML: https://mlsanity.com/paper/2211.14042/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 25 Nov 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Pre-training Molecular Graph Representation with 3D Geometry](https://mlsanity.com/paper/2110.07728/) (2021)
- [Molecular Graph Convolutions: Moving Beyond Fingerprints](https://mlsanity.com/paper/1603.00856/) (2016)
- [Drug discovery with explainable artificial intelligence](https://mlsanity.com/paper/2007.00523/) (2020)
- [N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules](https://mlsanity.com/paper/1806.09206/) (2018)
- [SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery](https://mlsanity.com/paper/1911.04738/) (2019)
- [MoCL: Data-driven Molecular Fingerprint via Knowledge-aware Contrastive Learning from Molecular Graph](https://mlsanity.com/paper/2106.04509/) (2021)
- [GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction](https://mlsanity.com/paper/2109.11730/) (2021)
- [Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery](https://mlsanity.com/paper/2012.01981/) (2020)
- [FP-GNN: a versatile deep learning architecture for enhanced molecular property prediction](https://mlsanity.com/paper/2205.03834/) (2022)
- [Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discovery](https://mlsanity.com/paper/2002.06053/) (2020)
- [GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animation](https://mlsanity.com/paper/1909.05729/) (2019)
- [Learning Attributed Graph Representations with Communicative Message Passing Transformer](https://mlsanity.com/paper/2107.08773/) (2021)
- [HamNet: Conformation-Guided Molecular Representation with Hamiltonian Neural Networks](https://mlsanity.com/paper/2105.03688/) (2021)
- [Attention-wise masked graph contrastive learning for predicting molecular property](https://mlsanity.com/paper/2206.08262/) (2022)
- [CheMixNet: Mixed DNN Architectures for Predicting Chemical Properties using Multiple Molecular Representations](https://mlsanity.com/paper/1811.08283/) (2018)

## Built on by (most cited first, 8 of 8)

- [ProtoMol: Enhancing Molecular Property Prediction via Prototype-Guided Multimodal Learning](https://mlsanity.com/paper/2510.16824/) (2025)
- [Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Model](https://mlsanity.com/paper/2408.09896/) (2024)
- [Dual-Modality Representation Learning for Molecular Property Prediction](https://mlsanity.com/paper/2501.06608/) (2025)
- [MOLAR: Learning Multimodal Molecular Representations from Noisy Labels](https://mlsanity.com/paper/2606.18390/) (2026)
- [Adapting Differential Molecular Representation with Hierarchical Prompts for Multi-label Property Prediction](https://mlsanity.com/paper/2405.18724/) (2024)
- [Molecular Graph Representation Learning via Structural Similarity Information](https://mlsanity.com/paper/2409.08580/) (2024)
- [DMMRL: Disentangled Multi-Modal Representation Learning via Variational Autoencoders for Molecular Property Prediction](https://mlsanity.com/paper/2603.21108/) (2026)
- [MoTIF-X: A Multimodal Tokenized Framework for Interpretable and Extensible Molecular Representation Learning](https://mlsanity.com/paper/2609.37384/) (2026)

## Abstract (arXiv)

In recent years, artificial intelligence has played an important role on accelerating the whole process of drug discovery. Various of molecular representation schemes of different modals (e.g. textual sequence or graph) are developed. By digitally encoding them, different chemical information can be learned through corresponding network structures. Molecular graphs and Simplified Molecular Input Line Entry System (SMILES) are popular means for molecular representation learning in current. Previous works have done attempts by combining both of them to solve the problem of specific information loss in single-modal representation on various tasks. To further fusing such multi-modal imformation, the correspondence between learned chemical feature from different representation should be considered. To realize this, we propose a novel framework of molecular joint representation learning via Multi-Modal information of SMILES and molecular Graphs, called MMSG. We improve the self-attention mechanism by introducing bond level graph representation as attention bias in Transformer to reinforce feature correspondence between multi-modal information. We further propose a Bidirectional Message Communication Graph Neural Network (BMC GNN) to strengthen the information flow aggregated from graphs for further combination. Numerous experiments on public property prediction datasets have demonstrated the effectiveness of our model.