# RotLSTM: Rotating Memories in Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2105.00357 · HTML: https://mlsanity.com/paper/2105.00357/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 21 references that are in the map.
- First posted on arXiv 1 May 2021.

## Builds on (most cited first, 17 of 17)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [End-to-End Differentiable Proving](https://mlsanity.com/paper/1705.11040/) (2017)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)
- [Visual Question Answering: A Survey of Methods and Datasets](https://mlsanity.com/paper/1607.05910/) (2016)
- [Ask Me Anything: Dynamic Memory Networks for Natural Language Processing](https://mlsanity.com/paper/1506.07285/) (2015)
- [Deep Voice: Real-time Neural Text-to-Speech](https://mlsanity.com/paper/1702.07825/) (2017)
- [Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs](https://mlsanity.com/paper/1612.05231/) (2016)
- [Programming with a Differentiable Forth Interpreter](https://mlsanity.com/paper/1605.06640/) (2016)
- [Gated Orthogonal Recurrent Units: On Learning to Forget](https://mlsanity.com/paper/1706.02761/) (2017)
- [Sequence-to-sequence neural network models for transliteration](https://mlsanity.com/paper/1610.09565/) (2016)
- [SCAN: Learning Hierarchical Compositional Visual Concepts](https://mlsanity.com/paper/1707.03389/) (2017)
- [Rotational Unit of Memory](https://mlsanity.com/paper/1710.09537/) (2017)

## Built on by (most cited first, 1 of 1)

- [RotRNN: Modelling Long Sequences with Rotations](https://mlsanity.com/paper/2407.07239/) (2024)

## Abstract (arXiv)

Long Short-Term Memory (LSTM) units have the ability to memorise and use long-term dependencies between inputs to generate predictions on time series data. We introduce the concept of modifying the cell state (memory) of LSTMs using rotation matrices parametrised by a new set of trainable weights. This addition shows significant increases of performance on some of the tasks from the bAbI dataset.