mlsanityData to 7 Oct 2026

Using pretrained graph neural networks with token mixers as geometric featurizers for conformational dynamics

arXiv 2409.19838 ↗First posted 30 Sep 2024cs.LG physics.chem-ph physics.comp-phcited by 5

Not on the map. Why?

Abstract (arXiv)

Identifying informative low-dimensional features that characterize dynamics in molecular simulations remains a challenge, often requiring extensive manual tuning and system-specific knowledge. Here, we introduce geom2vec, in which pretrained graph neural networks (GNNs) are used as universal geometric featurizers. By pretraining equivariant GNNs on a large dataset of molecular conformations with a self-supervised denoising objective, we obtain transferable structural representations that are useful for learning conformational dynamics without further fine-tuning. We show how the learned GNN representations can capture interpretable relationships between structural units (tokens) by combining them with expressive token mixers. Importantly, decoupling training the GNNs from training for downstream tasks enables analysis of larger molecular graphs (such as small proteins at all-atom resolution) with limited computational resources. In these ways, geom2vec eliminates the need for manual feature selection and increases the robustness of simulation analyses.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 41 of its 82 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

41 of 82 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2017Graph Attention Networkscited by 8,854
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  8. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  9. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  10. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  11. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  12. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
29 more
  1. 2020Directional Message Passing for Molecular Graphscited by 556
  2. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 438
  3. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  4. 2020Learning from Protein Structure with Geometric Vector Perceptronscited by 308
  5. 2019Cormorant: Covariant Molecular Neural Networkscited by 262
  6. 2017VAE with a VampPriorcited by 247
  7. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 179
  8. 2022Protein Representation Learning by Geometric Structure Pretrainingcited by 161
  9. 2017VAMPnets: Deep learning of molecular kineticscited by 135
  10. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  11. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  12. 2024Scaling Exponents Across Parameterizations and Optimizerscited by 71
  13. 2020Coarse Graining Molecular Dynamics with Graph Neural Networkscited by 56
  14. 2022Learning Hierarchical Protein Representations via Complete 3D Graph Networkscited by 54
  15. 2017Variational approach for learning Markov processes from time series datacited by 51
  16. 2022Structure-aware Protein Self-supervised Learningcited by 33
  17. 2022Contrastive Representation Learning for 3D Protein Structurescited by 30
  18. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  19. 2017Variational Encoding of Complex Dynamicscited by 21
  20. 2024Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fieldscited by 18
  21. 2024Evaluating representation learning on the protein structure universecited by 16
  22. 2019Nonlinear Discovery of Slow Molecular Modes using State-Free Reversible VAMPnetscited by 14
  23. 2024Pre-training with Fractional Denoising to Enhance Molecular Property Predictioncited by 14
  24. 2019Graph Dynamical Networks for Unsupervised Learning of Atomic Scale Dynamics in Materialscited by 13
  25. 2019High-resolution Markov state models for the dynamics of Trp-cage miniprotein constructed over slow folding modes identified by state-free reversible VAMPnetscited by 11
  26. 2023Inexact iterative numerical linear algebra for neural network-based spectral estimation and rare-event predictioncited by 9
  27. 2018Variational Selection of Features for Molecular Kineticscited by 8
  28. 2022GraphVAMPNet, using graph neural networks and variational approach to markov processes for dynamical modeling of biomoleculescited by 8
  29. 2023Transformers are efficient hierarchical chemical graph learnerscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Universally Converging Representations of Matter Across Scientific Foundation Modelscited by 12
  2. 2025An exact multiple-time-step variational formulation for the committor and the transition ratecited by 1
  3. 2025Hierarchical geometric deep learning enables scalable analysis of molecular dynamicscited by 1
  4. 2025Latent Thermodynamic Flows: Unified Representation Learning and Generative Modeling of Temperature-Dependent Behaviors from Limited Datacited by 0
  5. 2026Machine learning kinetics from molecular dynamics datacited by 0