mlsanityData to 14 Oct 2026

MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectra

arXiv 2502.16284 ↗First posted 22 Feb 2025cs.LG cs.AI cs.CEcited by 8

Not on the map. Why?

Abstract (arXiv)

Establishing the relationship between 3D structures and the energy states of molecular systems has proven to be a promising approach for learning 3D molecular representations. However, existing methods are limited to modeling the molecular energy states from classical mechanics. This limitation results in a significant oversight of quantum mechanical effects, such as quantized (discrete) energy level structures, which offer a more accurate estimation of molecular energy and can be experimentally measured through energy spectra. In this paper, we propose to utilize the energy spectra to enhance the pre-training of 3D molecular representations (MolSpectra), thereby infusing the knowledge of quantum mechanics into the molecular representations. Specifically, we propose SpecFormer, a multi-spectrum encoder for encoding molecular spectra via masked patch reconstruction. By further aligning outputs from the 3D encoder and spectrum encoder using a contrastive objective, we enhance the 3D encoder's understanding of molecules. Evaluations on public benchmarks reveal that our pre-trained representations surpass existing methods in predicting molecular properties and modeling dynamics.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 50 of its 72 references that are in the map. First posted on arXiv 22 Feb 2025.

Builds on

50 of 72 references.

References · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  6. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  7. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  8. 2021E(n) Equivariant Graph Neural Networkscited by 965
  9. 2020Directional Message Passing for Molecular Graphscited by 561
  10. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 474
  11. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  12. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 441
38 more
  1. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 264
  2. 2022Learning Local Equivariant Representations for Large-Scale Atomistic Dynamicscited by 256
  3. 2020Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Moleculescited by 255
  4. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  5. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 182
  6. 2021Spherical Message Passing for 3D Graph Networkscited by 168
  7. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  8. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  9. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  10. 2022Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matchingcited by 68
  11. 2021GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Predictioncited by 61
  12. 2023GSLB: The Graph Structure Learning Benchmarkcited by 50
  13. 2023Rethinking Tokenizer and Decoder in Masked Graph Modeling for Moleculescited by 49
  14. 2023A new perspective on building efficient and expressive 3D equivariant graph neural networkscited by 47
  15. 2022Unified 2D and 3D Pre-Training of Molecular Representationscited by 45
  16. 2022Energy-Motivated Equivariant Pretraining for 3D Molecular Graphscited by 37
  17. 2023Fractional Denoising for 3D Molecular Pre-trainingcited by 33
  18. 2023A Group Symmetric Stochastic Differential Equation Model for Molecule Multi-modal Pretrainingcited by 32
  19. 2024Text-Guided Molecule Generation with Diffusion Language Modelcited by 31
  20. 2024Unraveling Molecular Structure: A Multimodal Spectroscopic Dataset for Chemistrycited by 31
  21. 2018Rapid Prediction of Electron-Ionization Mass Spectrometry using Neural Networkscited by 28
  22. 2023Automated 3D Pre-Training for Molecular Property Predictioncited by 24
  23. 2024RAGraph: A General Retrieval-Augmented Graph Learning Frameworkcited by 24
  24. 2023Uncovering Neural Scaling Laws in Molecular Representation Learningcited by 23
  25. 2024Rethinking Graph Masked Autoencoders through Alignment and Uniformitycited by 22
  26. 2022Graph Self-supervised Learning with Accurate Discrepancy Learningcited by 21
  27. 2023Multimodal Molecular Pretraining via Modality Blendingcited by 21
  28. 2025Diffusion Models for Molecules: A Survey of Methods and Taskscited by 19
  29. 2023Sliced Denoising: A Physics-Informed Molecular Pre-Training Methodcited by 18
  30. 2023Prefix-Tree Decoding for Predicting Mass Spectra from Moleculescited by 16
  31. 2021Perspectives and Prospects on Transformer Architecture for Cross-Modal Tasks with Language and Visioncited by 14
  32. 2023Molecule Joint Auto-Encoding: Trajectory Pretraining with 2D and 3D Diffusioncited by 10
  33. 2024Beyond Efficiency: Molecular Data Pruning for Enhanced Generalizationcited by 9
  34. 2023Learning Harmonic Molecular Representations on Riemannian Manifoldcited by 8
  35. 2024DIVE: Subgraph Disagreement for Graph Out-of-Distribution Generalizationcited by 7
  36. 2023GeoTMI:Predicting quantum chemical property with easy-to-obtain geometry via positional denoisingcited by 5
  37. 2023Mass Spectra Prediction with Structural Motif-based Graph Neural Networkscited by 5
  38. 2024Pin-Tuning: Parameter-Efficient In-Context Tuning for Few-Shot Molecular Property Predictioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 1.

8 papers.

  1. 2025A Survey of Graph Transformers: Architectures, Theories and Applicationscited by 14
  2. 2025DiffSpectra: Molecular Structure Elucidation from Spectra using Diffusion Modelscited by 5
  3. 2025Atomic Diffusion Models for Small Molecule Structure Elucidation from NMR Spectracited by 4
  4. 2025GeoRecon: Graph-Level Representation Learning for 3D Molecules via Reconstruction-Based Pretrainingcited by 3
  5. 2025SpectrumWorld: Artificial Intelligence Foundation for Spectroscopycited by 2
  6. 2026MAST: Motif-Augmented Diffusion with Search Tree for Spectroscopic Molecular Structure Elucidationcited by 1
  7. 2025MolGA: Molecular Graph Adaptation with Pre-trained 2D Graph Encodercited by 0
  8. 20253D-GSRD: 3D Molecular Graph Auto-Encoder with Selective Re-mask Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Tokenizing 3D Molecule Structure with Quantized Spherical Coordinates
  2. 20233D Denoisers are Good 2D Teachers: Molecular Pretraining via Denoising and Cross-Modal Distillation
  3. 2025MolSculpt: Sculpting 3D Molecular Geometries from Chemical Syntax
  4. 2026VecMol: Vector-Field Representations for 3D Molecule Generation
  5. 2025D3MES: Diffusion Transformer with multihead equivariant self-attention for 3D molecule generation
  6. 2022Coordinating Cross-modal Distillation for Molecular Property Prediction
  7. 2025All-atom Diffusion Transformers: Unified generative modelling of molecules and materials
  8. 2026ED-DiT: Physics-Guided Diffusion Pretraining for Transferable Molecular Representations from Electron Density