mlsanityData to 14 Oct 2026

Multilingual Molecular Representation Learning via Contrastive Pre-training

arXiv 2109.08830 ↗First posted 18 Sep 2021cs.LG cs.CL physics.chem-phcited by 12

Not on the map. Why?

Abstract (arXiv)

Molecular representation learning plays an essential role in cheminformatics. Recently, language model-based approaches have gained popularity as an alternative to traditional expert-designed features to encode molecules. However, these approaches only utilize a single molecular language for representation learning. Motivated by the fact that a given molecule can be described using different languages such as Simplified Molecular Line Entry System (SMILES), The International Union of Pure and Applied Chemistry (IUPAC), and The IUPAC International Chemical Identifier (InChI), we propose a multilingual molecular embedding generation approach called MM-Deacon (multilingual molecular domain embedding analysis via contrastive learning). MM-Deacon is pre-trained using SMILES and IUPAC as two different languages on large-scale molecules. We evaluated the robustness of our method on seven molecular property prediction tasks from MoleculeNet benchmark, zero-shot cross-lingual retrieval, and a drug-drug interaction prediction task.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 25 of its 60 references that are in the map. First posted on arXiv 18 Sep 2021.

Builds on

25 of 60 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2016Layer Normalizationcited by 5,669
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  6. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  7. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  8. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  9. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  10. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
  11. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
  12. 2020Self-Supervised MultiModal Versatile Networkscited by 276
13 more
  1. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
  2. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
  3. 2019SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discoverycited by 90
  4. 2020Mapping the Space of Chemical Reactions Using Attention-Based Neural Networkscited by 70
  5. 2021Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videoscited by 62
  6. 2020Parameter Efficient Multimodal Transformers for Video Representation Learningcited by 52
  7. 2020P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understandingcited by 41
  8. 2017SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Propertiescited by 30
  9. 2021Dual-view Molecule Pre-trainingcited by 27
  10. 2020Contrastive Visual-Linguistic Pretrainingcited by 14
  11. 2020Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learningcited by 8
  12. 2018Latent Molecular Optimization for Targeted Therapeutic Designcited by 6
  13. 2020BERT Learns (and Teaches) Chemistrycited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 6; 2025: 1.

12 papers.

  1. 2023Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Languagecited by 44
  2. 2022A Systematic Survey of Chemical Pre-trained Modelscited by 34
  3. 2023Fractional Denoising for 3D Molecular Pre-trainingcited by 33
  4. 2024From Words to Molecules: A Survey of Large Language Models in Chemistrycited by 14
  5. 2024Pre-training with Fractional Denoising to Enhance Molecular Property Predictioncited by 14
  6. 2024ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modelingcited by 7
  7. 2024PolyCL: Contrastive Learning for Polymer Representation Learning via Explicit and Implicit Augmentationscited by 7
  8. 2022Multilingual and Multimodal Topic Modelling with Pretrained Embeddingscited by 6
  9. 2023UniMAP: Universal SMILES-Graph Representation Learningcited by 6
  10. 2024MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text Promptscited by 6
  11. 2024MaskMol: Knowledge-guided Molecular Image Pre-Training Framework for Activity Cliffscited by 2
  12. 2025Dual-Modality Representation Learning for Molecular Property Predictioncited by 1