# Transfer learning for chemically accurate interatomic neural network potentials

arXiv: https://arxiv.org/abs/2212.03916 · HTML: https://mlsanity.com/paper/2212.03916/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 71 references that are in the map.
- First posted on arXiv 7 Dec 2022.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Machine Learning Force Fields](https://mlsanity.com/paper/2010.07067/) (2020)
- [Self-Supervised Learning of Graph Neural Networks: A Unified Review](https://mlsanity.com/paper/2102.10757/) (2021)
- [Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive](https://mlsanity.com/paper/2105.07342/) (2021)
- [Less is more: sampling chemical space with active learning](https://mlsanity.com/paper/1801.09319/) (2018)
- [Does GNN Pretraining Help Molecular Representation?](https://mlsanity.com/paper/2207.06010/) (2022)
- [A Framework and Benchmark for Deep Batch Active Learning for Regression](https://mlsanity.com/paper/2203.09410/) (2022)
- [Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priors](https://mlsanity.com/paper/2205.10279/) (2022)
- [Machine Learning Molecular Dynamics for the Simulation of Infrared Spectra](https://mlsanity.com/paper/1705.05907/) (2017)
- [Fast Uncertainty Estimates in Deep Learning Interatomic Potentials](https://mlsanity.com/paper/2211.09866/) (2022)
- [Gaussian Moments as Physically Inspired Molecular Descriptors for Accurate and Scalable Machine Learning Potentials](https://mlsanity.com/paper/2109.07421/) (2021)
- [Fast and Sample-Efficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaussian Moments](https://mlsanity.com/paper/2109.09569/) (2021)
- [Automated Fitting of Neural Network Potentials at Coupled Cluster Accuracy: Protonated Water Clusters as Testing Ground](https://mlsanity.com/paper/1908.08734/) (2019)
- [Supervised Pretraining for Molecular Force Fields and Properties Prediction](https://mlsanity.com/paper/2211.14429/) (2022)
- [Synthetic data enable experiments in atomistic machine learning](https://mlsanity.com/paper/2211.16443/) (2022)

## Built on by (most cited first, 11 of 11)

- [Evaluation of the MACE Force Field Architecture: from Medicinal Chemistry to Materials Science](https://mlsanity.com/paper/2305.14247/) (2023)
- [Multitask methods for predicting molecular properties from heterogeneous data](https://mlsanity.com/paper/2401.17898/) (2024)
- [Synthetic pre-training for neural-network interatomic potentials](https://mlsanity.com/paper/2307.15714/) (2023)
- [Transfer learning for atomistic simulations using GNNs and kernel mean embeddings](https://mlsanity.com/paper/2306.01589/) (2023)
- [Environmental effects on emergent strategy in micro-scale multi-agent reinforcement learning](https://mlsanity.com/paper/2307.00994/) (2023)
- [Transfer Learning for Molecular Property Predictions from Small Data Sets](https://mlsanity.com/paper/2404.13393/) (2024)
- [Physics-Informed Weakly Supervised Learning for Interatomic Potentials](https://mlsanity.com/paper/2408.05215/) (2024)
- [Optimal Invariant Bases for Atomistic Machine Learning](https://mlsanity.com/paper/2503.23515/) (2025)
- [Implicit Delta Learning of High Fidelity Neural Network Potentials](https://mlsanity.com/paper/2412.06064/) (2024)
- [On-the-Fly Fine-Tuning of Foundational Neural Network Potentials: A Bayesian Neural Network Approach](https://mlsanity.com/paper/2507.13805/) (2025)
- [Transferable Implicit Solvent Machine Learning Potential for Drugs and Proteins Approaching Ab Initio Accuracy](https://mlsanity.com/paper/2607.10887/) (2026)

## Abstract (arXiv)

Developing machine learning-based interatomic potentials from ab-initio electronic structure methods remains a challenging task for computational chemistry and materials science. This work studies the capability of transfer learning, in particular discriminative fine-tuning, for efficiently generating chemically accurate interatomic neural network potentials on organic molecules from the MD17 and ANI data sets. We show that pre-training the network parameters on data obtained from density functional calculations considerably improves the sample efficiency of models trained on more accurate ab-initio data. Additionally, we show that fine-tuning with energy labels alone can suffice to obtain accurate atomic forces and run large-scale atomistic simulations, provided a well-designed fine-tuning data set. We also investigate possible limitations of transfer learning, especially regarding the design and size of the pre-training and fine-tuning data sets. Finally, we provide GM-NN potentials pre-trained and fine-tuned on the ANI-1x and ANI-1ccx data sets, which can easily be fine-tuned on and applied to organic molecules.