mlsanityData to 14 Oct 2026

Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching

arXiv 2206.13602 ↗First posted 27 Jun 2022cs.LG q-bio.QMcited by 68

Not on the map. Why?

Abstract (arXiv)

Molecular representation pretraining is critical in various applications for drug and material discovery due to the limited number of labeled molecules, and most existing work focuses on pretraining on 2D molecular graphs. However, the power of pretraining on 3D geometric structures has been less explored. This is owing to the difficulty of finding a sufficient proxy task that can empower the pretraining to effectively extract essential features from the geometric structures. Motivated by the dynamic nature of 3D molecules, where the continuous motion of a molecule in the 3D Euclidean space forms a smooth potential energy surface, we propose GeoSSL, a 3D coordinate denoising pretraining framework to model such an energy landscape. Further by leveraging an SE(3)-invariant score matching method, we propose GeoSSL-DDM in which the coordinate denoising proxy task is effectively boiled down to denoising the pairwise atomic distances in a molecule. Our comprehensive experiments confirm the effectiveness and robustness of our proposed method.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Builds on 52 of its 80 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

52 of 80 references.

References · 52

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  10. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  11. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  12. 2021E(n) Equivariant Graph Neural Networkscited by 965
38 more
  1. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 754
  2. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  3. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  4. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  5. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 571
  6. 2019Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Datacited by 532
  7. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  8. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 441
  9. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
  10. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 343
  11. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
  12. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
  13. 2020GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generationcited by 305
  14. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 303
  15. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  16. 2021GFlowNet Foundationscited by 290
  17. 2021Graph Self-Supervised Learning: A Surveycited by 289
  18. 2021How to Train Your Energy-Based Modelscited by 274
  19. 2020Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Moleculescited by 255
  20. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  21. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 243
  22. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 228
  23. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  24. 2020MoFlow: An Invertible Flow Model for Generating Molecular Graphscited by 185
  25. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  26. 2021Learning Gradient Fields for Molecular Conformation Generationcited by 163
  27. 2020Improved Contrastive Divergence Training of Energy Based Modelscited by 130
  28. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  29. 2021Geometric Deep Learning on Molecular Representationscited by 113
  30. 2018N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Moleculescited by 93
  31. 2020A Graph to Graphs Framework for Retrosynthesis Predictioncited by 86
  32. 2021GraphEBM: Molecular Graph Generation with Energy-Based Modelscited by 85
  33. 2020ATOM3D: Tasks On Molecules in Three Dimensionscited by 74
  34. 2022MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Designcited by 66
  35. 2020Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learningcited by 57
  36. 2021Rotation Invariant Graph Neural Networks using Spin Convolutionscited by 52
  37. 2020Learning Graph-Convolutional Representations for Point Cloud Denoisingcited by 40
  38. 2021Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphscited by 31

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 21; 2024: 21; 2025: 13; 2026: 6.

68 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 166
  3. 2023GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learningcited by 70
  4. 2023A Text-guided Protein Design Frameworkcited by 64
  5. 2023Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasetscited by 39
  6. 2023From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Predictioncited by 39
  7. 2022A Systematic Survey of Chemical Pre-trained Modelscited by 34
  8. 2023Fractional Denoising for 3D Molecular Pre-trainingcited by 33
  9. 2023A Group Symmetric Stochastic Differential Equation Model for Molecule Multi-modal Pretrainingcited by 32
  10. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  11. 2023DiffPack: A Torsional Diffusion Model for Autoregressive Protein Side-Chain Packingcited by 24
  12. 2023Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materialscited by 22
38 more
  1. 2022Graph Neural Networks for Moleculescited by 21
  2. 2022Few-shot Image Generation with Diffusion Modelscited by 21
  3. 2023Multimodal Molecular Pretraining via Modality Blendingcited by 21
  4. 2023Sliced Denoising: A Physics-Informed Molecular Pre-Training Methodcited by 18
  5. 2024Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fieldscited by 18
  6. 2024UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learningcited by 18
  7. 2022Geometry-Complete Perceptron Networks for 3D Molecular Graphscited by 16
  8. 2024Evaluating representation learning on the protein structure universecited by 16
  9. 2023DomainStudio: Fine-Tuning Diffusion Models for Domain-Driven Image Generation using Limited Datacited by 15
  10. 2023Protein-ligand binding representation learning from fine-grained interactionscited by 14
  11. 2024Pre-training with Fractional Denoising to Enhance Molecular Property Predictioncited by 14
  12. 2023Pre-Training Protein Encoder via Siamese Sequence-Structure Diffusion Trajectory Predictioncited by 13
  13. 2023ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignmentcited by 13
  14. 2024Geometric Representation Condition Improves Equivariant Molecule Generationcited by 13
  15. 2024Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformerscited by 12
  16. 2023Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentialscited by 11
  17. 2023Molecule Joint Auto-Encoding: Trajectory Pretraining with 2D and 3D Diffusioncited by 10
  18. 2024Molecular relaxation by reverse diffusion with time step predictioncited by 9
  19. 2024Beyond Efficiency: Molecular Data Pruning for Enhanced Generalizationcited by 9
  20. 2025MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectracited by 8
  21. 2024ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modelingcited by 7
  22. 2024MolTailor: Tailoring Chemical Molecular Representation to Specific Tasks via Text Promptscited by 6
  23. 2024Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Surveycited by 6
  24. 2023GeoTMI:Predicting quantum chemical property with easy-to-obtain geometry via positional denoisingcited by 5
  25. 2024An Equivariant Pretrained Transformer for Unified 3D Molecular Representation Learningcited by 5
  26. 2025Rao-Blackwell Gradient Estimators for Equivariant Denoising Diffusioncited by 5
  27. 2024SubGDiff: A Subgraph Diffusion Model to Improve Molecular Representation Learningcited by 4
  28. 2025Equivariant Masked Position Prediction for Efficient Molecular Representationcited by 4
  29. 2026Self-Conditioned Denoising for Atomistic Representation Learningcited by 4
  30. 2023A Crystal-Specific Pre-Training Framework for Crystal Material Property Predictioncited by 3
  31. 2024Multi-Type Point Cloud Autoencoder: A Complete Equivariant Embedding for Molecule Conformation and Posecited by 3
  32. 2024Pushing the Limits of All-Atom Geometric Graph Neural Networks: Pre-Training, Scaling and Zero-Shot Transfercited by 3
  33. 2025Flow Along the K-Amplitude for Generative Modelingcited by 3
  34. 2025GeoRecon: Graph-Level Representation Learning for 3D Molecules via Reconstruction-Based Pretrainingcited by 3
  35. 2025TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondencecited by 3
  36. 2022Flaky Performances when Pretraining on Relational Databasescited by 2
  37. 2024Atomas: Hierarchical Alignment on Molecule-Text for Unified Molecule Understanding and Generationcited by 2
  38. 2024GeomCLIP: Contrastive Geometry-Text Pre-training for Moleculescited by 2

50 most cited of 68 All 68 (JSON)