# Conditional updates of neural network weights for increased out of training performance

arXiv: https://arxiv.org/abs/2512.03653 · HTML: https://mlsanity.com/paper/2512.03653/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 48 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Builds on (most cited first, 7 of 7)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [Will Artificial Intelligence supersede Earth System and Climate Models?](https://mlsanity.com/paper/2101.09126/) (2021)
- [Climate-Invariant Machine Learning](https://mlsanity.com/paper/2112.08440/) (2021)
- [Improving predictions by nonlinear regression models from outlying input data](https://mlsanity.com/paper/2003.07926/) (2020)
- [Forecasting the Future with Yesterday's Climate: Temperature Bias in AI Weather and Climate Models](https://mlsanity.com/paper/2509.22359/) (2025)

## Abstract (arXiv)

This study proposes a method to enhance neural network performance when training data and application data are not very similar, e.g., out of distribution problems, as well as pattern and regime shifts. The method consists of three main steps: 1) Retrain the neural network towards reasonable subsets of the training data set and note down the resulting weight anomalies. 2) Choose reasonable predictors and derive a regression between the predictors and the weight anomalies. 3) Extrapolate the weights, and thereby the neural network, to the application data. We show and discuss this method in three use cases from the climate sciences, which include successful temporal, spatial and cross-domain extrapolations of neural networks.