mlsanityData to 7 Oct 2026

3D Infomax improves GNNs for Molecular Property Prediction

arXiv 2110.04126 ↗First posted 8 Oct 2021cs.LG cs.AI q-bio.BMcited by 146

Not on the map. Why?

Abstract (arXiv)

Molecular property prediction is one of the fastest-growing applications of deep learning with critical real-world impacts. Including 3D molecular structure as input to learned models improves their performance for many molecular tasks. However, this information is infeasible to compute at the scale required by several real-world applications. We propose pre-training a model to reason about the geometry of molecules given only their 2D molecular graphs. Using methods from self-supervised learning, we maximize the mutual information between 3D summary vectors and the representations of a Graph Neural Network (GNN) such that they contain latent 3D information. During fine-tuning on molecules with unknown geometry, the GNN still generates implicit 3D information and can use it to improve downstream tasks. We show that 3D pre-training provides significant improvements for a wide range of properties, such as a 22% average MAE reduction on eight quantum mechanical properties. Moreover, the learned representations can be effectively transferred between datasets in different molecular spaces.

Key facts. Cited by 146 papers in the map (data through 7 Oct 2026). Builds on 38 of its 77 references that are in the map. First posted on arXiv 8 Oct 2021.

Builds on

38 of 77 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  9. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  10. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  11. 2018On the Spectral Bias of Neural Networkscited by 1,321
  12. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
26 more
  1. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  2. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  3. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  4. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  5. 2021E(n) Equivariant Graph Neural Networkscited by 951
  6. 2019Contrastive Representation Distillationcited by 734
  7. 2018MINE: Mutual Information Neural Estimationcited by 680
  8. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 569
  9. 2020Directional Message Passing for Molecular Graphscited by 556
  10. 2020Principal Neighbourhood Aggregation for Graph Netscited by 477
  11. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 373
  12. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 342
  13. 2021Graph Contrastive Learning Automatedcited by 287
  14. 2020Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Moleculescited by 254
  15. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  16. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 238
  17. 2021Learning Gradient Fields for Molecular Conformation Generationcited by 162
  18. 2021Self-supervised Graph-level Representation Learning with Local and Global Structurecited by 125
  19. 2021GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensemblescited by 106
  20. 2021Learning Neural Generative Dynamics for Molecular Conformation Generationcited by 87
  21. 2019GNN-FiLM: Graph Neural Networks with Feature-wise Linear Modulationcited by 75
  22. 2019Reconstructing continuous distributions of 3D protein structure from cryo-EM imagescited by 56
  23. 2017Learning Graph-Level Representation for Drug Discoverycited by 43
  24. 2020Molecular machine learning with conformer ensemblescited by 27
  25. 2021Dual-view Molecule Pre-trainingcited by 27
  26. 2019Utilizing Edge Features in Graph Neural Networks via Variational Information Maximizationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 21; 2023: 51; 2024: 35; 2025: 24; 2026: 14.

146 papers.

Sort:

By area:

  1. 2022Long Range Graph Benchmarkcited by 258
  2. 2023MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generationcited by 96
  3. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  4. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  5. 2023GFlowNets for AI-Driven Scientific Discoverycited by 63
  6. 2022Graph-based Molecular Representation Learningcited by 51
  7. 2023MolFM: A Multimodal Molecular Foundation Modelcited by 50
  8. 2022Unified 2D and 3D Pre-Training of Molecular Representationscited by 45
  9. 2023Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Languagecited by 44
  10. 2021On Representation Knowledge Distillation for Graph Neural Networkscited by 38
  11. 2023Knowledge Distillation on Graphs: A Surveycited by 38
  12. 2023Large Language Models for Scientific Synthesis, Inference and Explanationcited by 38
38 more
  1. 2022Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrastcited by 37
  2. 2022Energy-Motivated Equivariant Pretraining for 3D Molecular Graphscited by 37
  3. 2023Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasetscited by 37
  4. 2024On the Scalability of GNNs for Molecular Graphscited by 36
  5. 2022Crystal Twins: Self-supervised Learning for Crystalline Material Property Predictioncited by 34
  6. 2022A Systematic Survey of Chemical Pre-trained Modelscited by 34
  7. 2023Fractional Denoising for 3D Molecular Pre-trainingcited by 33
  8. 2023A Group Symmetric Stochastic Differential Equation Model for Molecule Multi-modal Pretrainingcited by 32
  9. 2022KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Predictioncited by 31
  10. 2023Graph Positional and Structural Encodercited by 31
  11. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  12. 2023Automated 3D Pre-Training for Molecular Property Predictioncited by 24
  13. 2022Latent Graph Inference using Product Manifoldscited by 23
  14. 2022Evaluating Self-Supervised Learning for Molecular Graph Embeddingscited by 22
  15. 2023Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+cited by 22
  16. 2023Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materialscited by 22
  17. 2022Graph Neural Networks for Moleculescited by 21
  18. 2023Multimodal Molecular Pretraining via Modality Blendingcited by 21
  19. 2023Fragment-based Pretraining and Finetuning on Molecular Graphscited by 20
  20. 2024Uni-Mol2: Exploring Molecular Pretraining Model at Scalecited by 20
  21. 2024UniGEM: A Unified Approach to Generation and Property Prediction for Moleculescited by 20
  22. 2022BARTSmiles: Generative Masked Language Models for Molecular Representationscited by 18
  23. 2023Generalist Equivariant Transformer Towards 3D Molecular Interaction Learningcited by 18
  24. 2023Sliced Denoising: A Physics-Informed Molecular Pre-Training Methodcited by 18
  25. 2024UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learningcited by 18
  26. 2024MolParser: End-to-end Visual Recognition of Molecule Structures in the Wildcited by 17
  27. 2025NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generationcited by 17
  28. 2022DPA-1: Pretraining of Attention-based Deep Potential Model for Molecular Simulationcited by 15
  29. 2024Pre-training with Fractional Denoising to Enhance Molecular Property Predictioncited by 14
  30. 2024TopoGCL: Topological Graph Contrastive Learningcited by 13
  31. 2022Graph Neural Networks in Particle Physics: Implementations, Innovations, and Challengescited by 12
  32. 2022CoSP: Co-supervised pretraining of pocket and ligandcited by 12
  33. 2024Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformerscited by 12
  34. 2023Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentialscited by 11
  35. 2023MUBen: Benchmarking the Uncertainty of Molecular Representation Modelscited by 11
  36. 2024Pre-training Graph Neural Networks on Molecules by Using Subgraph-Conditioned Graph Information Bottleneckcited by 11
  37. 2023Neural Atoms: Propagating Long-range Interaction in Molecular Graphs through Efficient Communication Channelcited by 10
  38. 2023Identifying Semantic Component for Robust Molecular Property Predictioncited by 10

50 most cited of 146 All 146 (JSON)