# Pre-training with Fractional Denoising to Enhance Molecular Property Prediction

arXiv: https://arxiv.org/abs/2407.11086 · HTML: https://mlsanity.com/paper/2407.11086/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 96 references that are in the map.
- First posted on arXiv 14 Jul 2024.

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [Equivariant message passing for the prediction of tensorial properties and molecular spectra](https://mlsanity.com/paper/2102.03150/) (2021)
- [Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules](https://mlsanity.com/paper/2011.14115/) (2020)
- [Pre-training Molecular Graph Representation with 3D Geometry](https://mlsanity.com/paper/2110.07728/) (2021)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction](https://mlsanity.com/paper/2106.06130/) (2021)
- [Semi-supervised Sequence Learning](https://mlsanity.com/paper/1511.01432/) (2015)
- [Random Features Strengthen Graph Neural Networks](https://mlsanity.com/paper/2002.03155/) (2020)
- [TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials](https://mlsanity.com/paper/2202.02541/) (2022)
- [DeepDTA: Deep Drug-Target Binding Affinity Prediction](https://mlsanity.com/paper/1801.10193/) (2018)
- [Spherical Message Passing for 3D Graph Networks](https://mlsanity.com/paper/2102.05013/) (2021)
- [Motif-based Graph Self-Supervised Learning for Molecular Property Prediction](https://mlsanity.com/paper/2110.00987/) (2021)
- [3D Infomax improves GNNs for Molecular Property Prediction](https://mlsanity.com/paper/2110.04126/) (2021)
- [Pre-training via Denoising for Molecular Property Prediction](https://mlsanity.com/paper/2206.00133/) (2022)
- [One Transformer Can Understand Both 2D & 3D Molecular Data](https://mlsanity.com/paper/2210.01765/) (2022)
- [ATOM3D: Tasks On Molecules in Three Dimensions](https://mlsanity.com/paper/2012.04035/) (2020)
- [Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching](https://mlsanity.com/paper/2206.13602/) (2022)
- [GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction](https://mlsanity.com/paper/2109.11730/) (2021)
- [Robust Optimization as Data Augmentation for Large-scale Graphs](https://mlsanity.com/paper/2010.09891/) (2020)
- [Learning Hierarchical Protein Representations via Complete 3D Graph Networks](https://mlsanity.com/paper/2207.12600/) (2022)

## Built on by (most cited first, 14 of 14)

- [UniGEM: A Unified Approach to Generation and Property Prediction for Molecules](https://mlsanity.com/paper/2410.10516/) (2024)
- [Geometric Representation Condition Improves Equivariant Molecule Generation](https://mlsanity.com/paper/2410.03655/) (2024)
- [Relaxed Equivariance via Multitask Learning](https://mlsanity.com/paper/2410.17878/) (2024)
- [Using pretrained graph neural networks with token mixers as geometric featurizers for conformational dynamics](https://mlsanity.com/paper/2409.19838/) (2024)
- [Self-Conditioned Denoising for Atomistic Representation Learning](https://mlsanity.com/paper/2603.17196/) (2026)
- [DecoyDB: A Dataset for Graph Contrastive Learning in Protein-Ligand Binding Affinity Prediction](https://mlsanity.com/paper/2507.06366/) (2025)
- [Two-Stage Pretraining for Molecular Property Prediction in the Wild](https://mlsanity.com/paper/2411.03537/) (2024)
- [Pretraining a Foundation Model for Small-Molecule Natural Products](https://mlsanity.com/paper/2503.17656/) (2025)
- [AlphaFold Database Debiasing for Robust Inverse Folding](https://mlsanity.com/paper/2506.08365/) (2025)
- [Iterative Pretraining Framework for Interatomic Potentials](https://mlsanity.com/paper/2507.20118/) (2025)
- [Unified Molecule Pre-training with Flexible 2D and 3D Modalities: Single and Paired Modality Integration](https://mlsanity.com/paper/2510.07035/) (2025)
- [A Conformation-Centric Generative Foundation Model for Linear Polymer Modeling and Design](https://mlsanity.com/paper/2510.16023/) (2025)
- [Chemistry-Enhanced Diffusion-Based Framework for Small-to-Large Molecular Conformation Generation](https://mlsanity.com/paper/2511.12182/) (2025)
- [Decoding Mixture Perception through Computational Modeling of Component Interactions](https://mlsanity.com/paper/2609.11958/) (2026)

## Abstract (arXiv)

Deep learning methods have been considered promising for accelerating molecular screening in drug discovery and material design. Due to the limited availability of labelled data, various self-supervised molecular pre-training methods have been presented. While many existing methods utilize common pre-training tasks in computer vision (CV) and natural language processing (NLP), they often overlook the fundamental physical principles governing molecules. In contrast, applying denoising in pre-training can be interpreted as an equivalent force learning, but the limited noise distribution introduces bias into the molecular distribution. To address this issue, we introduce a molecular pre-training framework called fractional denoising (Frad), which decouples noise design from the constraints imposed by force learning equivalence. In this way, the noise becomes customizable, allowing for incorporating chemical priors to significantly improve molecular distribution modeling. Experiments demonstrate that our framework consistently outperforms existing methods, establishing state-of-the-art results across force prediction, quantum chemical properties, and binding affinity tasks. The refined noise design enhances force accuracy and sampling coverage, which contribute to the creation of physically consistent molecular representations, ultimately leading to superior predictive performance.