mlsanityData to 7 Oct 2026

Molecule Joint Auto-Encoding: Trajectory Pretraining with 2D and 3D Diffusion

arXiv 2312.03475 ↗First posted 6 Dec 2023cs.LG cs.AI q-bio.BMcited by 10

Not on the map. Why?

Abstract (arXiv)

Recently, artificial intelligence for drug discovery has raised increasing interest in both machine learning and chemistry domains. The fundamental building block for drug discovery is molecule geometry and thus, the molecule's geometrical representation is the main bottleneck to better utilize machine learning techniques for drug discovery. In this work, we propose a pretraining method for molecule joint auto-encoding (MoleculeJAE). MoleculeJAE can learn both the 2D bond (topology) and 3D conformation (geometry) information, and a diffusion process model is applied to mimic the augmented trajectories of such two modalities, based on which, MoleculeJAE will learn the inherent chemical structure in a self-supervised manner. Thus, the pretrained geometrical representation in MoleculeJAE is expected to benefit downstream geometry-related tasks. Empirically, MoleculeJAE proves its effectiveness by reaching state-of-the-art performance on 15 out of 20 tasks by comparing it with 12 competitive baselines.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 59 of its 74 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

59 of 74 references.

References · 59

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2023A Survey of Large Language Modelscited by 2,537
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  12. 2023Consistency Modelscited by 1,778
38 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  3. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  4. 2021E(n) Equivariant Graph Neural Networkscited by 951
  5. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  6. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  7. 2020Self-supervised Learning: Generative or Contrastivecited by 693
  8. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  9. 2021Score-based Generative Modeling in Latent Spacecited by 634
  10. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  11. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  12. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  13. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  14. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  15. 2021Graph Self-Supervised Learning: A Surveycited by 289
  16. 2023Effective Data Augmentation With Diffusion Modelscited by 282
  17. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
  18. 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equationscited by 273
  19. 2022Torsional Diffusion for Molecular Conformer Generationcited by 262
  20. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  21. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 238
  22. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 230
  23. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  24. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 182
  25. 2021Learning Gradient Fields for Molecular Conformation Generationcited by 162
  26. 20213D Infomax improves GNNs for Molecular Property Predictioncited by 146
  27. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 146
  28. 2020Bayesian Neural Networks: An Introduction and Surveycited by 130
  29. 2022gDDIM: Generalized denoising diffusion implicit modelscited by 125
  30. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  31. 2021Accurate Learning of Graph Representations with Graph Multiset Poolingcited by 120
  32. 2022Soft Diffusion: Score Matching for General Corruptionscited by 101
  33. 2022Image Data Augmentation for Deep Learning: A Surveycited by 96
  34. 2018N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Moleculescited by 93
  35. 2016Disentangling factors of variation in deep representations using adversarial trainingcited by 87
  36. 2022How Mask Matters: Towards Theoretical Understandings of Masked Autoencoderscited by 86
  37. 2021SE(3) Equivariant Graph Neural Networks with Complete Local Framescited by 75
  38. 2022Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matchingcited by 67

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 3; 2026: 1.

10 papers.

  1. 2024UniGEM: A Unified Approach to Generation and Property Prediction for Moleculescited by 20
  2. 2024UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learningcited by 18
  3. 2025MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectracited by 8
  4. 2024A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamicscited by 5
  5. 2024GeomCLIP: Contrastive Geometry-Text Pre-training for Moleculescited by 2
  6. 2024Manifold-Constrained Nucleus-Level Denoising Diffusion Model for Structure-Based Drug Designcited by 1
  7. 2024Sculpting Molecules in Text-3D Space: A Flexible Substructure Aware Framework for Text-Oriented Molecular Optimizationcited by 0
  8. 2025Unified Molecule Pre-training with Flexible 2D and 3D Modalities: Single and Paired Modality Integrationcited by 0
  9. 2025MolGA: Molecular Graph Adaptation with Pre-trained 2D Graph Encodercited by 0
  10. 2026Graph Energy Matching: Transport-Aligned Energy-Based Modeling for Graph Generationcited by 0