mlsanityData to 14 Oct 2026

DPA-1: Pretraining of Attention-based Deep Potential Model for Molecular Simulation

arXiv 2208.08236 ↗First posted 17 Aug 2022physics.chem-ph cs.LG physics.comp-phcited by 15

Not on the map. Why?

Abstract (arXiv)

Machine learning assisted modeling of the inter-atomic potential energy surface (PES) is revolutionizing the field of molecular simulation. With the accumulation of high-quality electronic structure data, a model that can be pretrained on all available data and finetuned on downstream tasks with a small additional effort would bring the field to a new stage. Here we propose DPA-1, a Deep Potential model with a novel attention mechanism, which is highly effective for representing the conformation and chemical spaces of atomic systems and learning the PES. We tested DPA-1 on a number of systems and observed superior performance compared with existing benchmarks. When pretrained on large-scale datasets containing 56 elements, DPA-1 can be successfully applied to various downstream tasks with a great improvement of sample efficiency. Surprisingly, for different elements, the learned type embedding parameters form a spiral in the latent space and have a natural correspondence with their positions on the periodic table, showing interesting interpretability of the pretrained DPA-1 model.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 28 of its 77 references that are in the map. First posted on arXiv 17 Aug 2022.

Builds on

28 of 77 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  8. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  9. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 571
  10. 2020Directional Message Passing for Molecular Graphscited by 561
  11. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 441
  12. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 343
16 more
  1. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 264
  2. 2022Learning Local Equivariant Representations for Large-Scale Atomistic Dynamicscited by 256
  3. 2020Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Moleculescited by 255
  4. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  5. 2020The Open Catalyst 2020 (OC20) Dataset and Community Challengescited by 246
  6. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  7. 2017Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanicscited by 214
  8. 2020Machine Learning Force Fieldscited by 207
  9. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  10. 2019Attention in Natural Language Processingcited by 101
  11. 2018Less is more: sampling chemical space with active learningcited by 82
  12. 2022GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasetscited by 61
  13. 2021Rotation Invariant Graph Neural Networks using Spin Convolutionscited by 52
  14. 2022Spherical Channels for Modeling Atomic Interactionscited by 47
  15. 2018Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulationcited by 37
  16. 2022Equivariant Graph Attention Networks for Molecular Property Predictioncited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 7; 2026: 4.

15 papers.

Sort:

By area:

  1. 2025MatLLMSearch: Crystal Structure Discovery with Evolution-Guided Large Language Modelscited by 7
  2. 2023Synthetic pre-training for neural-network interatomic potentialscited by 6
  3. 2024Overcoming systematic softening in universal machine learning interatomic potentials by fine-tuningcited by 6
  4. 2024Transfer Learning for Molecular Property Predictions from Small Data Setscited by 2
  5. 2025AlphaNet: Scaling Up Local-frame-based Atomistic Interatomic Potentialcited by 2
  6. 2026Scalable Machine Learning Force Fields for Macromolecular Systems Through Long-Range Aware Message Passingcited by 2
  7. 2026Better without U: Impact of Selective Hubbard U Correction on Foundational MLIPscited by 2
  8. 2025The OpenLAM Challengescited by 1
  9. 2025ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphscited by 1
  10. 2026Mixture of experts architectures for machine learning interatomic potentialscited by 1
  11. 2024Pretraining Strategy for Neural Potentialscited by 0
  12. 2025A General Neural Network Potential for Energetic Materials with C, H, N, and O elementscited by 0
3 more
  1. 2025Are Foundational Atomistic Models Reliable for Finite-Temperature Molecular Dynamics?cited by 0
  2. 2025Learning task-specific predictive models for scientific computingcited by 0
  3. 2026Making Room for AI: Multi-GPU Molecular Dynamics with Deep Potentials in GROMACScited by 0