# Time Series Prediction under Distribution Shift using Differentiable Forgetting

arXiv: https://arxiv.org/abs/2207.11486 · HTML: https://mlsanity.com/paper/2207.11486/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 23 Jul 2022.

## Builds on (most cited first, 5 of 5)

- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization](https://mlsanity.com/paper/1607.05447/) (2016)
- [Rethinking Importance Weighting for Deep Learning under Distribution Shift](https://mlsanity.com/paper/2006.04662/) (2020)

## Built on by (most cited first, 5 of 5)

- [Koopman Neural Forecaster for Time Series with Temporal Distribution Shifts](https://mlsanity.com/paper/2210.03675/) (2022)
- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [LOB-Based Deep Learning Models for Stock Price Trend Prediction: A Benchmark Study](https://mlsanity.com/paper/2308.01915/) (2023)
- [Optimal Empirical Risk Minimization under Temporal Distribution Shifts](https://mlsanity.com/paper/2507.13287/) (2025)
- [Using ARIMA to Predict the Expansion of Subscriber Data Consumption](https://mlsanity.com/paper/2404.15095/) (2024)

## Abstract (arXiv)

Time series prediction is often complicated by distribution shift which demands adaptive models to accommodate time-varying distributions. We frame time series prediction under distribution shift as a weighted empirical risk minimisation problem. The weighting of previous observations in the empirical risk is determined by a forgetting mechanism which controls the trade-off between the relevancy and effective sample size that is used for the estimation of the predictive model. In contrast to previous work, we propose a gradient-based learning method for the parameters of the forgetting mechanism. This speeds up optimisation and therefore allows more expressive forgetting mechanisms.