mlsanityData to 14 Oct 2026

Pre-training Molecular Graph Representation with 3D Geometry

arXiv 2110.07728 ↗First posted 7 Oct 2021cs.LG cs.CV eess.IVcited by 248

Not on the map. Why?

Abstract (arXiv)

Molecular graph representation learning is a fundamental problem in modern drug and material discovery. Molecular graphs are typically modeled by their 2D topological structures, but it has been recently discovered that 3D geometric information plays a more vital role in predicting molecular functionalities. However, the lack of 3D information in real-world scenarios has significantly impeded the learning of geometric graph representation. To cope with this challenge, we propose the Graph Multi-View Pre-training (GraphMVP) framework where self-supervised learning (SSL) is performed by leveraging the correspondence and consistency between 2D topological structures and 3D geometric views. GraphMVP effectively learns a 2D molecular graph encoder that is enhanced by richer and more discriminative 3D geometry. We further provide theoretical insights to justify the effectiveness of GraphMVP. Finally, comprehensive experiments show that GraphMVP can consistently outperform existing graph SSL methods.

Key facts. Cited by 248 papers in the map (data through 14 Oct 2026). Builds on 58 of its 108 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

58 of 108 references.

References · 58

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2017Inductive Representation Learning on Large Graphscited by 6,320
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2018How Powerful are Graph Neural Networks?cited by 4,393
  10. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
38 more
  1. 2020Supervised Contrastive Learningcited by 2,829
  2. 2016Density estimation using Real NVPcited by 2,042
  3. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  4. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  5. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  6. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  7. 2018Deep Graph Infomaxcited by 1,106
  8. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  9. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  10. 2021E(n) Equivariant Graph Neural Networkscited by 965
  11. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  12. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  13. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  14. 2018MINE: Mutual Information Neural Estimationcited by 683
  15. 2020Contrastive Multi-View Representation Learning on Graphscited by 652
  16. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  17. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 571
  18. 2019On Variational Bounds of Mutual Informationcited by 559
  19. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  20. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 506
  21. 2020Principal Neighbourhood Aggregation for Graph Netscited by 478
  22. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  23. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
  24. 2019On Mutual Information Maximization for Representation Learningcited by 316
  25. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 303
  26. 2021Graph Self-Supervised Learning: A Surveycited by 289
  27. 2021Graph Contrastive Learning Automatedcited by 287
  28. 2021How to Train Your Energy-Based Modelscited by 274
  29. 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 256
  30. 2016Learning Representations for Automatic Colorizationcited by 251
  31. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 243
  32. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  33. 2018DeepDTA: Deep Drug-Target Binding Affinity Predictioncited by 173
  34. 2020Improved Contrastive Divergence Training of Energy Based Modelscited by 130
  35. 2021Self-supervised Graph-level Representation Learning with Local and Global Structurecited by 125
  36. 2018N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Moleculescited by 93
  37. 2020AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversariescited by 90
  38. 2020Generalized Energy Based Modelscited by 73

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 37; 2023: 56; 2024: 75; 2025: 50; 2026: 28.

248 papers.

Sort:

By area:

  1. 2022Long Range Graph Benchmarkcited by 263
  2. 2022Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pocketscited by 183
  3. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 166
  4. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  5. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  6. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  7. 2022Diffusion-based Molecule Generation with Informative Prior Bridgescited by 106
  8. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103
  9. 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
  10. 2023MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generationcited by 98
  11. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  12. 2023GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Textcited by 82
38 more
  1. 2023MolXPT: Wrapping Molecules with Text for Generative Pre-trainingcited by 81
  2. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  3. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  4. 2023GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learningcited by 70
  5. 2022Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matchingcited by 68
  6. 2022MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Designcited by 66
  7. 2024Towards 3D Molecule-Text Interpretation in Language Modelscited by 66
  8. 2023A Text-guided Protein Design Frameworkcited by 64
  9. 2022Does GNN Pretraining Help Molecular Representation?cited by 60
  10. 2022Let Invariant Rationale Discovery Inspire Graph Contrastive Learningcited by 57
  11. 2024BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuningcited by 54
  12. 2022Graph-based Molecular Representation Learningcited by 51
  13. 2023MolFM: A Multimodal Molecular Foundation Modelcited by 50
  14. 2023Rethinking Tokenizer and Decoder in Masked Graph Modeling for Moleculescited by 49
  15. 2022Unified 2D and 3D Pre-Training of Molecular Representationscited by 45
  16. 2023Can Large Language Models Empower Molecular Property Prediction?cited by 44
  17. 2023SELFormer: Molecular Representation Learning via SELFIES Language Modelscited by 40
  18. 2023Large Language Models for Scientific Synthesis, Inference and Explanationcited by 39
  19. 2023From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Predictioncited by 39
  20. 2021On Representation Knowledge Distillation for Graph Neural Networkscited by 38
  21. 2022Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrastcited by 37
  22. 2022Energy-Motivated Equivariant Pretraining for 3D Molecular Graphscited by 37
  23. 2024On the Scalability of GNNs for Molecular Graphscited by 36
  24. 2022A Systematic Survey of Chemical Pre-trained Modelscited by 34
  25. 2023Fractional Denoising for 3D Molecular Pre-trainingcited by 33
  26. 2023A Group Symmetric Stochastic Differential Equation Model for Molecule Multi-modal Pretrainingcited by 32
  27. 2022KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Predictioncited by 31
  28. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  29. 2023Automated 3D Pre-Training for Molecular Property Predictioncited by 24
  30. 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Predictioncited by 24
  31. 2022A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applicationscited by 23
  32. 2023Uncovering Neural Scaling Laws in Molecular Representation Learningcited by 23
  33. 2024UniMoT: Unified Molecule-Text Language Model with Discrete Token Representationcited by 23
  34. 2022Evaluating Self-Supervised Learning for Molecular Graph Embeddingscited by 22
  35. 2023Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+cited by 22
  36. 2023Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materialscited by 22
  37. 2022Graph Neural Networks for Moleculescited by 21
  38. 2023Multimodal Molecular Pretraining via Modality Blendingcited by 21

50 most cited of 248 All 248 (JSON)