# Implicit Delta Learning of High Fidelity Neural Network Potentials

arXiv: https://arxiv.org/abs/2412.06064 · HTML: https://mlsanity.com/paper/2412.06064/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 60 references that are in the map.
- First posted on arXiv 8 Dec 2024.

## Builds on (most cited first, 25 of 25)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [Scaling Laws for Transfer](https://mlsanity.com/paper/2102.01293/) (2021)
- [Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations](https://mlsanity.com/paper/2210.07237/) (2022)
- [Less is more: sampling chemical space with active learning](https://mlsanity.com/paper/1801.09319/) (2018)
- [SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentials](https://mlsanity.com/paper/2209.10702/) (2022)
- [Machine Learning of coarse-grained Molecular Dynamics Force Fields](https://mlsanity.com/paper/1812.01736/) (2018)
- [OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features](https://mlsanity.com/paper/2007.08026/) (2020)
- [DScribe: Library of Descriptors for Machine Learning in Materials Science](https://mlsanity.com/paper/1904.08875/) (2019)
- [From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction](https://mlsanity.com/paper/2310.16802/) (2023)
- [Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation](https://mlsanity.com/paper/1810.11890/) (2018)
- [Scaling the leading accuracy of deep equivariant models to biomolecular simulations of realistic size](https://mlsanity.com/paper/2304.10061/) (2023)
- [Learning neural network potentials from experimental data via Differentiable Trajectory Reweighting](https://mlsanity.com/paper/2106.01138/) (2021)
- [Hyperactive Learning (HAL) for Data-Driven Interatomic Potentials](https://mlsanity.com/paper/2210.04225/) (2022)
- [TorchMD-Net 2.0: Fast Neural Network Potentials for Molecular Simulations](https://mlsanity.com/paper/2402.17660/) (2024)
- [PubChemQC B3LYP/6-31G*//PM6 dataset: the Electronic Structures of 86 Million Molecules using B3LYP/6-31G* calculations](https://mlsanity.com/paper/2305.18454/) (2023)
- [Transfer learning for chemically accurate interatomic neural network potentials](https://mlsanity.com/paper/2212.03916/) (2022)
- [Scalable Bayesian Uncertainty Quantification for Neural Network Potentials: Promise and Pitfalls](https://mlsanity.com/paper/2212.07959/) (2022)
- [Synthetic pre-training for neural-network interatomic potentials](https://mlsanity.com/paper/2307.15714/) (2023)
- [Transfer Learning for Molecular Property Predictions from Small Data Sets](https://mlsanity.com/paper/2404.13393/) (2024)

## Abstract (arXiv)

Neural network potentials (NNPs) offer a fast and accurate alternative to ab-initio methods for molecular dynamics (MD) simulations but are hindered by the high cost of training data from high-fidelity Quantum Mechanics (QM) methods. Our work introduces the Implicit Delta Learning (IDLe) method, which reduces the need for high-fidelity QM data by leveraging cheaper semi-empirical QM computations without compromising NNP accuracy or inference cost. IDLe employs an end-to-end multi-task architecture with fidelity-specific heads that decode energies based on a shared latent representation of the input atomistic system. In various settings, IDLe achieves the same accuracy as single high-fidelity baselines while using up to 50x less high-fidelity data. This result could significantly reduce data generation cost and consequently enhance accuracy and generalization, and expand chemical coverage for NNPs, advancing MD simulations for material science and drug discovery. Additionally, we provide a novel set of 11 million semi-empirical QM calculations to support future multi-fidelity NNP modeling.