# May the Force be with You: Unified Force-Centric Pre-Training for 3D Molecular Conformations

arXiv: https://arxiv.org/abs/2308.14759 · HTML: https://mlsanity.com/paper/2308.14759/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [GemNet: Universal Directional Graph Neural Networks for Molecules](https://mlsanity.com/paper/2106.08903/) (2021)
- [OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs](https://mlsanity.com/paper/2103.09430/) (2021)
- [Machine Learning Force Fields](https://mlsanity.com/paper/2010.07067/) (2020)
- [TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials](https://mlsanity.com/paper/2202.02541/) (2022)
- [Learning Gradient Fields for Molecular Conformation Generation](https://mlsanity.com/paper/2105.03902/) (2021)
- [Pre-training via Denoising for Molecular Property Prediction](https://mlsanity.com/paper/2206.00133/) (2022)
- [Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations](https://mlsanity.com/paper/2210.07237/) (2022)
- [Simple GNN Regularisation for 3D Molecular Property Prediction & Beyond](https://mlsanity.com/paper/2106.07971/) (2021)
- [Energy-Motivated Equivariant Pretraining for 3D Molecular Graphs](https://mlsanity.com/paper/2207.08824/) (2022)
- [Supervised Pretraining for Molecular Force Fields and Properties Prediction](https://mlsanity.com/paper/2211.14429/) (2022)

## Built on by (most cited first, 13 of 13)

- [From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction](https://mlsanity.com/paper/2310.16802/) (2023)
- [Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fields](https://mlsanity.com/paper/2403.09549/) (2024)
- [Pre-training with Fractional Denoising to Enhance Molecular Property Prediction](https://mlsanity.com/paper/2407.11086/) (2024)
- [UniSim: A Unified Simulator for Time-Coarsened Dynamics of Biomolecules](https://mlsanity.com/paper/2506.03157/) (2025)
- [UniMAP: Universal SMILES-Graph Representation Learning](https://mlsanity.com/paper/2310.14216/) (2023)
- [Self-Conditioned Denoising for Atomistic Representation Learning](https://mlsanity.com/paper/2603.17196/) (2026)
- [PolyConf: Unlocking Polymer Conformation Generation through Hierarchical Generative Models](https://mlsanity.com/paper/2504.08859/) (2025)
- [Elign: Equivariant Diffusion Model Alignment from Foundational Machine Learning Force Fields](https://mlsanity.com/paper/2601.21985/) (2026)
- [Stable Autonomous Flow Matching](https://mlsanity.com/paper/2402.05774/) (2024)
- [Pretraining Strategy for Neural Potentials](https://mlsanity.com/paper/2402.15921/) (2024)
- [Hessian-Informed Flow Matching](https://mlsanity.com/paper/2410.11433/) (2024)
- [A Conformation-Centric Generative Foundation Model for Linear Polymer Modeling and Design](https://mlsanity.com/paper/2510.16023/) (2025)
- [Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials](https://mlsanity.com/paper/2605.08988/) (2026)

## Abstract (arXiv)

Recent works have shown the promise of learning pre-trained models for 3D molecular representation. However, existing pre-training models focus predominantly on equilibrium data and largely overlook off-equilibrium conformations. It is challenging to extend these methods to off-equilibrium data because their training objective relies on assumptions of conformations being the local energy minima. We address this gap by proposing a force-centric pretraining model for 3D molecular conformations covering both equilibrium and off-equilibrium data. For off-equilibrium data, our model learns directly from their atomic forces. For equilibrium data, we introduce zero-force regularization and forced-based denoising techniques to approximate near-equilibrium forces. We obtain a unified pre-trained model for 3D molecular representation with over 15 million diverse conformations. Experiments show that, with our pre-training objective, we increase forces accuracy by around 3 times compared to the un-pre-trained Equivariant Transformer model. By incorporating regularizations on equilibrium data, we solved the problem of unstable MD simulations in vanilla Equivariant Transformers, achieving state-of-the-art simulation performance with 2.45 times faster inference time than NequIP. As a powerful molecular encoder, our pre-trained model achieves on-par performance with state-of-the-art property prediction tasks.