mlsanityData to 14 Oct 2026

Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model

arXiv 2211.10590 ↗First posted 19 Nov 2022cs.LG cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

The recent success of large foundation models in artificial intelligence has prompted the emergence of chemical pre-trained models. Despite the growing interest in large molecular pre-trained models that provide informative representations for downstream tasks, attempts for multimodal pre-training approaches on the molecule domain were limited. To address this, we present a novel multimodal molecular pre-trained model that incorporates the modalities of structure and biochemical properties, drawing inspiration from recent advances in multimodal learning techniques. Our proposed model pipeline of data handling and training objectives aligns the structure/property features in a common embedding space, which enables the model to regard bidirectional information between the molecules' structure and properties. These contributions emerge synergistic knowledge, allowing us to tackle both multimodal and unimodal downstream tasks through a single model. Through extensive experiments, we demonstrate that our model shows remarkable capabilities in solving various meaningful chemical challenges, including conditional molecule generation, property prediction, molecule classification, and reaction prediction.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 38 of its 75 references that are in the map. First posted on arXiv 19 Nov 2022.

Builds on

38 of 75 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  11. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  12. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
26 more
  1. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  2. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  3. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  4. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  5. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
  6. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  7. 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 267
  8. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
  9. 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 206
  10. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  11. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 200
  12. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  13. 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 127
  14. 2017Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Networkcited by 103
  15. 2018Multi-Objective De Novo Drug Design with Conditional Graph Generative Modelcited by 101
  16. 2018N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Moleculescited by 93
  17. 2020Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Editscited by 88
  18. 2018Molecular generative model based on conditional variational autoencoder for de novo molecular designcited by 73
  19. 2021Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction predictioncited by 68
  20. 2017SMILES Enumeration as Data Augmentation for Neural Network Modeling of Moleculescited by 59
  21. 2022polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informaticscited by 48
  22. 2022TransPolymer: a Transformer-based language model for polymer property predictionscited by 43
  23. 2020PIGNet: A physics-informed deep learning model toward generalized drug-target interaction predictionscited by 31
  24. 2021Dual-view Molecule Pre-trainingcited by 27
  25. 2022Accurate, reliable and interpretable solubility prediction of druglike molecules with attention pooling and Bayesian learningcited by 3
  26. 2022Can Pre-trained Models Really Learn Better Molecular Representations for AI-aided Drug Discovery?cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 10; 2026: 5.

25 papers.

Sort:

By area:

  1. 2024ChemLLM: A Chemical Large Language Modelcited by 106
  2. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  3. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  4. 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Modelscited by 11
  5. 2024Beyond Efficiency: Molecular Data Pruning for Enhanced Generalizationcited by 9
  6. 2024A Large Encoder-Decoder Family of Foundation Models For Chemical Languagecited by 5
  7. 2026Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materialscited by 5
  8. 2024Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networkscited by 4
  9. 2024A Physics-Inspired Deep Learning Framework with Polar Coordinate Attention for Ptychographic Imagingcited by 4
  10. 2024Property Enhanced Instruction Tuning for Multi-task Molecule Generation with Large Language Modelscited by 3
  11. 2025AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Researchcited by 3
  12. 2026Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correctioncited by 3
13 more
  1. 2025A Foundation Model for Material Fracture Predictioncited by 2
  2. 2025AttriLens-Mol: Attribute Guided Reinforcement Learning for Molecular Property Prediction with Large Language Modelscited by 2
  3. 2026Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligencecited by 2
  4. 2025Toward a Robust and Generalizable Metamaterial Foundation Modelcited by 1
  5. 2026AROMMA: Unifying Olfactory Embeddings for Single Molecules and Mixturescited by 1
  6. 2024Accelerating Drug Safety Assessment using Bidirectional-LSTM for SMILES Datacited by 0
  7. 2024Efficient Training of Transformers for Molecule Property Prediction on Small-scale Datasetscited by 0
  8. 2025Multimodal 3D Genome Pre-trainingcited by 0
  9. 2025Integrating Single-Cell Foundation Models with Graph Neural Networks for Drug Response Predictioncited by 0
  10. 2025Small Data Explainer -- The impact of small data methods in everyday lifecited by 0
  11. 2025Cross-Modality Controlled Molecule Generation with Diffusion Language Modelcited by 0
  12. 2025Hierarchical Structure-Property Alignment for Data-Efficient Molecular Generation and Editingcited by 0
  13. 2026Higher-Order Molecular Grammars for Generative and Foundation Models in Chemistrycited by 0