# Reconstructing Kernel-based Machine Learning Force Fields with Super-linear Convergence

arXiv: https://arxiv.org/abs/2212.12737 · HTML: https://mlsanity.com/paper/2212.12737/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 84 references that are in the map.
- First posted on arXiv 24 Dec 2022.

## Builds on (most cited first, 24 of 24)

- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Machine Learning Force Fields](https://mlsanity.com/paper/2010.07067/) (2020)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Machine learning for molecular simulation](https://mlsanity.com/paper/1911.02792/) (2019)
- [SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects](https://mlsanity.com/paper/2105.00304/) (2021)
- [FALKON: An Optimal Large Scale Kernel Method](https://mlsanity.com/paper/1705.10958/) (2017)
- [Revisiting the Nystrom Method for Improved Large-Scale Machine Learning](https://mlsanity.com/paper/1303.1849/) (2013)
- [Faster Kernel Ridge Regression Using Sketching and Preconditioning](https://mlsanity.com/paper/1611.03220/) (2016)
- [So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems](https://mlsanity.com/paper/2205.14276/) (2022)
- [Hierarchical modeling of molecular energies using a deep neural network](https://mlsanity.com/paper/1710.00017/) (2017)
- [Preconditioning Kernel Matrices](https://mlsanity.com/paper/1602.06693/) (2016)
- [Uniform Sampling for Matrix Approximation](https://mlsanity.com/paper/1408.5099/) (2014)
- [Diving into the shallows: a computational perspective on large-scale shallow learning](https://mlsanity.com/paper/1703.10622/) (2017)
- [Improving CUR Matrix Decomposition and the Nystr"om Approximation via Adaptive Sampling](https://mlsanity.com/paper/1303.4207/) (2013)
- [Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes](https://mlsanity.com/paper/1706.02524/) (2017)
- [Fast DPP Sampling for Nystr"om with Application to Kernel Methods](https://mlsanity.com/paper/1603.06052/) (2016)
- [Provable Deterministic Leverage Score Sampling](https://mlsanity.com/paper/1404.1530/) (2014)
- [Ridge Regression and Provable Deterministic Ridge Leverage Score Sampling](https://mlsanity.com/paper/1803.06010/) (2018)
- [Accurate Machine Learned Quantum-Mechanical Force Fields for Biomolecular Simulations](https://mlsanity.com/paper/2205.08306/) (2022)
- [Algorithmic Differentiation for Automated Modeling of Machine Learned Force Fields](https://mlsanity.com/paper/2208.12104/) (2022)
- [Fast and Accurate Gaussian Kernel Ridge Regression Using Matrix Decompositions for Preconditioning](https://mlsanity.com/paper/1905.10587/) (2019)
- [BIGDML: Towards Exact Machine Learning Force Fields for Materials](https://mlsanity.com/paper/2106.04229/) (2021)

## Built on by (most cited first, 2 of 2)

- [Robust, randomized preconditioning for kernel ridge regression](https://mlsanity.com/paper/2304.12465/) (2023)
- [From Peptides to Nanostructures: A Euclidean Transformer for Fast and Stable Machine Learned Force Fields](https://mlsanity.com/paper/2309.15126/) (2023)

## Abstract (arXiv)

Kernel machines have sustained continuous progress in the field of quantum chemistry. In particular, they have proven to be successful in the low-data regime of force field reconstruction. This is because many equivariances and invariances due to physical symmetries can be incorporated into the kernel function to compensate for much larger datasets. So far, the scalability of kernel machines has however been hindered by its quadratic memory and cubical runtime complexity in the number of training points. While it is known, that iterative Krylov subspace solvers can overcome these burdens, their convergence crucially relies on effective preconditioners, which are elusive in practice. Effective preconditioners need to partially pre-solve the learning problem in a computationally cheap and numerically robust manner. Here, we consider the broad class of Nystr"om-type methods to construct preconditioners based on successively more sophisticated low-rank approximations of the original kernel matrix, each of which provides a different set of computational trade-offs. All considered methods aim to identify a representative subset of inducing (kernel) columns to approximate the dominant kernel spectrum.