mlsanityData to 7 Oct 2026

Pretraining Strategy for Neural Potentials

arXiv 2402.15921 ↗First posted 24 Feb 2024cs.LG physics.chem-phcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a mask pretraining method for Graph Neural Networks (GNNs) to improve their performance on fitting potential energy surfaces, particularly in water systems. GNNs are pretrained by recovering spatial information related to masked-out atoms from molecules, then transferred and finetuned on atomic forcefields. Through such pretraining, GNNs learn meaningful prior about structural and underlying physical information of molecule systems that are useful for downstream tasks. From comprehensive experiments and ablation studies, we show that the proposed method improves the accuracy and convergence speed compared to GNNs trained from scratch or using other pretraining techniques such as denoising. On the other hand, our pretraining method is suitable for both energy-centric and force-centric GNNs. This approach showcases its potential to enhance the performance and data efficiency of GNNs in fitting molecular force fields.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 88 references that are in the map. First posted on arXiv 24 Feb 2024.

Builds on

41 of 88 references.

References · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  3. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  4. 2021E(n) Equivariant Graph Neural Networkscited by 951
  5. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  6. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  7. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
  8. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  9. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 569
  10. 2020Directional Message Passing for Molecular Graphscited by 556
  11. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
  12. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 342
29 more
  1. 2020Learning from Protein Structure with Geometric Vector Perceptronscited by 308
  2. 2021Graph Self-Supervised Learning: A Surveycited by 289
  3. 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 268
  4. 2019Cormorant: Covariant Molecular Neural Networkscited by 262
  5. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 258
  6. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  7. 2016Molecular Graph Convolutions: Moving Beyond Fingerprintscited by 238
  8. 2017Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanicscited by 213
  9. 2020Machine Learning Force Fieldscited by 204
  10. 2021Motif-based Graph Self-Supervised Learning for Molecular Property Predictioncited by 159
  11. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  12. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  13. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  14. 2019Machine learning for molecular simulationcited by 109
  15. 2021SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effectscited by 103
  16. 2017DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamicscited by 79
  17. 2023Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamicscited by 72
  18. 2022Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matchingcited by 67
  19. 2021ForceNet: A Graph Neural Network for Large-Scale Quantum Calculationscited by 58
  20. 2020TorchMD: A deep learning framework for molecular simulationscited by 55
  21. 2017Hierarchical modeling of molecular energies using a deep neural networkcited by 37
  22. 2022Improving Molecular Contrastive Learning via Faulty Negative Mitigation and Decomposed Fragment Contrastcited by 37
  23. 2022Crystal Twins: Self-supervised Learning for Crystalline Material Property Predictioncited by 34
  24. 2024Cartesian atomic cluster expansion for machine learning interatomic potentialscited by 31
  25. 2022MOFormer: Self-Supervised Transformer model for Metal-Organic Framework Property Predictioncited by 28
  26. 2022DPA-1: Pretraining of Attention-based Deep Potential Model for Molecular Simulationcited by 15
  27. 2023May the Force be with You: Unified Force-Centric Pre-Training for 3D Molecular Conformationscited by 13
  28. 2022GNS: A generalizable Graph Neural Network-based simulator for particulate and fluid modelingcited by 12
  29. 2023Denoise Pretraining on Nonequilibrium Molecules for Accurate and Transferable Neural Potentialscited by 11

Built on by

Not cited yet.