Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model
Not on the map. Why?
Abstract (arXiv)
The recent success of large foundation models in artificial intelligence has prompted the emergence of chemical pre-trained models. Despite the growing interest in large molecular pre-trained models that provide informative representations for downstream tasks, attempts for multimodal pre-training approaches on the molecule domain were limited. To address this, we present a novel multimodal molecular pre-trained model that incorporates the modalities of structure and biochemical properties, drawing inspiration from recent advances in multimodal learning techniques. Our proposed model pipeline of data handling and training objectives aligns the structure/property features in a common embedding space, which enables the model to regard bidirectional information between the molecules' structure and properties. These contributions emerge synergistic knowledge, allowing us to tackle both multimodal and unimodal downstream tasks through a single model. Through extensive experiments, we demonstrate that our model shows remarkable capabilities in solving various meaningful chemical challenges, including conditional molecule generation, property prediction, molecule classification, and reaction prediction.
Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 38 of its 75 references that are in the map. First posted on arXiv 19 Nov 2022.
Builds on
38 of 75 references.
References · 38
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
26 more
- 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
- 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
- 2019Strategies for Pre-training Graph Neural Networkscited by 971
- 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
- 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
- 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
- 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 267
- 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
- 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 206
- 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
- 2018Optimization of Molecules via Deep Reinforcement Learningcited by 200
- 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
- 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 127
- 2017Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Networkcited by 103
- 2018Multi-Objective De Novo Drug Design with Conditional Graph Generative Modelcited by 101
- 2018N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Moleculescited by 93
- 2020Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Editscited by 88
- 2018Molecular generative model based on conditional variational autoencoder for de novo molecular designcited by 73
- 2021Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction predictioncited by 68
- 2017SMILES Enumeration as Data Augmentation for Neural Network Modeling of Moleculescited by 59
- 2022polyBERT: A chemical language model to enable fully machine-driven ultrafast polymer informaticscited by 48
- 2022TransPolymer: a Transformer-based language model for polymer property predictionscited by 43
- 2020PIGNet: A physics-informed deep learning model toward generalized drug-target interaction predictionscited by 31
- 2021Dual-view Molecule Pre-trainingcited by 27
- 2022Accurate, reliable and interpretable solubility prediction of druglike molecules with attention pooling and Bayesian learningcited by 3
- 2022Can Pre-trained Models Really Learn Better Molecular Representations for AI-aided Drug Discovery?cited by 2
Built on by
Citing papers per year: 2024: 10; 2025: 10; 2026: 5.
25 papers.
Sort:
By area:
- 2024ChemLLM: A Chemical Large Language Modelcited by 106
- 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
- 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
- 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Modelscited by 11
- 2024Beyond Efficiency: Molecular Data Pruning for Enhanced Generalizationcited by 9
- 2024A Large Encoder-Decoder Family of Foundation Models For Chemical Languagecited by 5
- 2026Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materialscited by 5
- 2024Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networkscited by 4
- 2024A Physics-Inspired Deep Learning Framework with Polar Coordinate Attention for Ptychographic Imagingcited by 4
- 2024Property Enhanced Instruction Tuning for Multi-task Molecule Generation with Large Language Modelscited by 3
- 2025AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Researchcited by 3
- 2026Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correctioncited by 3
13 more
- 2025A Foundation Model for Material Fracture Predictioncited by 2
- 2025AttriLens-Mol: Attribute Guided Reinforcement Learning for Molecular Property Prediction with Large Language Modelscited by 2
- 2026Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligencecited by 2
- 2025Toward a Robust and Generalizable Metamaterial Foundation Modelcited by 1
- 2026AROMMA: Unifying Olfactory Embeddings for Single Molecules and Mixturescited by 1
- 2024Accelerating Drug Safety Assessment using Bidirectional-LSTM for SMILES Datacited by 0
- 2024Efficient Training of Transformers for Molecule Property Prediction on Small-scale Datasetscited by 0
- 2025Multimodal 3D Genome Pre-trainingcited by 0
- 2025Integrating Single-Cell Foundation Models with Graph Neural Networks for Drug Response Predictioncited by 0
- 2025Small Data Explainer -- The impact of small data methods in everyday lifecited by 0
- 2025Cross-Modality Controlled Molecule Generation with Diffusion Language Modelcited by 0
- 2025Hierarchical Structure-Property Alignment for Data-Efficient Molecular Generation and Editingcited by 0
- 2026Higher-Order Molecular Grammars for Generative and Foundation Models in Chemistrycited by 0
- 2026Higher-Order Molecular Grammars for Generative and Foundation Models in Chemistry1 Oct 2026
- 2026Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence12 Aug 2026
- 2026Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction4 Jun 2026
- 2026Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materials24 Feb 2026
- 2026AROMMA: Unifying Olfactory Embeddings for Single Molecules and Mixtures27 Jan 2026
- 2025Hierarchical Structure-Property Alignment for Data-Efficient Molecular Generation and Editing11 Nov 2025
- 2025Cross-Modality Controlled Molecule Generation with Diffusion Language Model20 Aug 2025
- 2025AttriLens-Mol: Attribute Guided Reinforcement Learning for Molecular Property Prediction with Large Language Models6 Aug 2025
- 2025A Foundation Model for Material Fracture Prediction30 Jul 2025
- 2025Small Data Explainer -- The impact of small data methods in everyday life15 Jul 2025
- 2025Toward a Robust and Generalizable Metamaterial Foundation Model3 Jul 2025
- 2025AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research17 May 2025
13 more
- 2025Integrating Single-Cell Foundation Models with Graph Neural Networks for Drug Response Prediction19 Apr 2025
- 2025Multimodal 3D Genome Pre-training12 Apr 2025
- 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning5 Feb 2025
- 2024Property Enhanced Instruction Tuning for Multi-task Molecule Generation with Large Language Models24 Dec 2024
- 2024A Physics-Inspired Deep Learning Framework with Polar Coordinate Attention for Ptychographic Imaging25 Nov 2024
- 2024Efficient Training of Transformers for Molecule Property Prediction on Small-scale Datasets7 Sep 2024
- 2024Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization2 Sep 2024
- 2024A Large Encoder-Decoder Family of Foundation Models For Chemical Language24 Jul 2024
- 2024Accelerating Drug Safety Assessment using Bidirectional-LSTM for SMILES Data8 Jul 2024
- 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery16 Jun 2024
- 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Models28 May 2024
- 2024ChemLLM: A Chemical Large Language Model10 Feb 2024
- 2024Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networks3 Feb 2024
- 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Modelscited by 11
- 2026Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correctioncited by 3
- 2025Cross-Modality Controlled Molecule Generation with Diffusion Language Modelcited by 0
- 2026Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correction4 Jun 2026
- 2025Cross-Modality Controlled Molecule Generation with Diffusion Language Model20 Aug 2025
- 2024LDMol: A Text-to-Molecule Diffusion Model with Structurally Informative Latent Space Surpasses AR Models28 May 2024
- 2024ChemLLM: A Chemical Large Language Modelcited by 106
- 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
- 2024Beyond Efficiency: Molecular Data Pruning for Enhanced Generalizationcited by 9
- 2024A Large Encoder-Decoder Family of Foundation Models For Chemical Languagecited by 5
- 2026Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materialscited by 5
- 2024Structure-Aware E(3)-Invariant Molecular Conformer Aggregation Networkscited by 4
- 2024A Physics-Inspired Deep Learning Framework with Polar Coordinate Attention for Ptychographic Imagingcited by 4
- 2024Property Enhanced Instruction Tuning for Multi-task Molecule Generation with Large Language Modelscited by 3
- 2025AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Researchcited by 3
- 2025A Foundation Model for Material Fracture Predictioncited by 2
- 2026Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligencecited by 2
- 2025Toward a Robust and Generalizable Metamaterial Foundation Modelcited by 1
12 most cited of 20 All 20 (JSON)
- 2026Higher-Order Molecular Grammars for Generative and Foundation Models in Chemistry1 Oct 2026
- 2026Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence12 Aug 2026
- 2026Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materials24 Feb 2026
- 2026AROMMA: Unifying Olfactory Embeddings for Single Molecules and Mixtures27 Jan 2026
- 2025Hierarchical Structure-Property Alignment for Data-Efficient Molecular Generation and Editing11 Nov 2025
- 2025A Foundation Model for Material Fracture Prediction30 Jul 2025
- 2025Small Data Explainer -- The impact of small data methods in everyday life15 Jul 2025
- 2025Toward a Robust and Generalizable Metamaterial Foundation Model3 Jul 2025
- 2025AI-Driven Automation Can Become the Foundation of Next-Era Science of Science Research17 May 2025
- 2025Integrating Single-Cell Foundation Models with Graph Neural Networks for Drug Response Prediction19 Apr 2025
- 2025Multimodal 3D Genome Pre-training12 Apr 2025
- 2024Property Enhanced Instruction Tuning for Multi-task Molecule Generation with Large Language Models24 Dec 2024
12 newest of 20 All 20 (JSON)