# Deep Learning with Kernel Flow Regularization for Time Series Forecasting

arXiv: https://arxiv.org/abs/2109.11649 · HTML: https://mlsanity.com/paper/2109.11649/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 6 of 6)

- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [A Survey of Deep Learning Techniques for Neural Machine Translation](https://mlsanity.com/paper/2002.07526/) (2020)
- [Kernel Flows: from learning kernels from data into the abyss](https://mlsanity.com/paper/1808.04475/) (2018)
- [Regularization of Deep Neural Networks with Spectral Dropout](https://mlsanity.com/paper/1711.08591/) (2017)
- [Deep regularization and direct training of the inner layers of Neural Networks with Kernel Flows](https://mlsanity.com/paper/2002.08335/) (2020)

## Built on by (most cited first, 1 of 1)

- [WaveBound: Dynamic Error Bounds for Stable Time Series Forecasting](https://mlsanity.com/paper/2210.14303/) (2022)

## Abstract (arXiv)

Long Short-Term Memory (LSTM) neural networks have been widely used for time series forecasting problems. However, LSTMs are prone to overfitting and performance reduction during test phases. Several different regularization techniques have been shown in literature to prevent overfitting problems in neural networks. In this paper, first, we introduce application of kernel flow methods for time series forecasting in general. Afterward, we examine the effectiveness of applying kernel flow regularization on LSTM layers to avoid overfitting problems. We describe a regularization method by applying kernel flow loss function on LSTM layers. In experimental results, we show that kernel flow outperforms baseline models on time series forecasting benchmarks. We also compare the effect of dropout and kernel flow regularization techniques on LSTMs. The experimental results illustrate that kernel flow achieves similar regularization effect to dropout. It also shows that the best results is obtained using both kernel flow and dropout regularizations with early stopping on LSTM layers on some time series datasets (e.g. power-load demand forecasts).