# Long short-term relevance learning

arXiv: https://arxiv.org/abs/2106.12694 · HTML: https://mlsanity.com/paper/2106.12694/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 43 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Probabilistic Recurrent State-Space Models](https://mlsanity.com/paper/1801.10395/) (2018)
- [Physics-Informed Multi-LSTM Networks for Metamodeling of Nonlinear Structures](https://mlsanity.com/paper/2002.10253/) (2020)
- [Bayesian Recurrent Neural Networks](https://mlsanity.com/paper/1704.02798/) (2017)
- [From Bayesian Sparsity to Gated Recurrent Nets](https://mlsanity.com/paper/1706.02815/) (2017)
- [An Approximate Bayesian Long Short-Term Memory Algorithm for Outlier Detection](https://mlsanity.com/paper/1712.08773/) (2017)
- [Stochastic state estimation via incremental iterative sparse polynomial chaos based Bayesian-Gauss-Newton-Markov-Kalman filter](https://mlsanity.com/paper/1909.07209/) (2019)

## Abstract (arXiv)

To incorporate prior knowledge as well as measurement uncertainties in the traditional long short term memory (LSTM) neural networks, an efficient sparse Bayesian training algorithm is introduced to the network architecture. The proposed scheme automatically determines relevant neural connections and adapts accordingly, in contrast to the classical LSTM solution. Due to its flexibility, the new LSTM scheme is less prone to overfitting, and hence can approximate time dependent solutions by use of a smaller data set. On a structural nonlinear finite element application we show that the self-regulating framework does not require prior knowledge of a suitable network architecture and size, while ensuring satisfying accuracy at reasonable computational cost.