# Lifelong Machine Learning Potentials

arXiv: https://arxiv.org/abs/2303.05911 · HTML: https://mlsanity.com/paper/2303.05911/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 97 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [Machine Learning Force Fields](https://mlsanity.com/paper/2010.07067/) (2020)
- [Memory Efficient Experience Replay for Streaming Learning](https://mlsanity.com/paper/1809.05922/) (2018)
- [Machine learning for molecular simulation](https://mlsanity.com/paper/1911.02792/) (2019)
- [Continuous Learning in Single-Incremental-Task Scenarios](https://mlsanity.com/paper/1806.08568/) (2018)
- [SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects](https://mlsanity.com/paper/2105.00304/) (2021)
- [Less is more: sampling chemical space with active learning](https://mlsanity.com/paper/1801.09319/) (2018)
- [Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization](https://mlsanity.com/paper/1805.10966/) (2018)
- [Machine learning for electronically excited states of molecules](https://mlsanity.com/paper/2007.05320/) (2020)
- [WACSF - Weighted Atom-Centered Symmetry Functions as Descriptors in Machine Learning Potentials](https://mlsanity.com/paper/1712.05861/) (2017)
- [Machine learning enables long time scale molecular photodynamics simulations](https://mlsanity.com/paper/1811.09112/) (2018)

## Built on by (most cited first, 3 of 3)

- [FeNNol: an Efficient and Flexible Library for Building Force-field-enhanced Neural Network Potentials](https://mlsanity.com/paper/2405.01491/) (2024)
- [NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance](https://mlsanity.com/paper/2408.08776/) (2024)
- [AURA: Angular Update Rate Adaptation for training complex-valued neural networks](https://mlsanity.com/paper/2609.25914/) (2026)

## Abstract (arXiv)

Machine learning potentials (MLPs) trained on accurate quantum chemical data can retain the high accuracy, while inflicting little computational demands. On the downside, they need to be trained for each individual system. In recent years, a vast number of MLPs has been trained from scratch because learning additional data typically requires to train again on all data to not forget previously acquired knowledge. Additionally, most common structural descriptors of MLPs cannot represent efficiently a large number of different chemical elements. In this work, we tackle these problems by introducing element-embracing atom-centered symmetry functions (eeACSFs) which combine structural properties and element information from the periodic table. These eeACSFs are a key for our development of a lifelong machine learning potential (lMLP). Uncertainty quantification can be exploited to transgress a fixed, pre-trained MLP to arrive at a continuously adapting lMLP, because a predefined level of accuracy can be ensured. To extend the applicability of an lMLP to new systems, we apply continual learning strategies to enable autonomous and on-the-fly training on a continuous stream of new data. For the training of deep neural networks, we propose the continual resilient (CoRe) optimizer and incremental learning strategies relying on rehearsal of data, regularization of parameters, and the architecture of the model.