# LiteLSTM Architecture for Deep Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2201.11624 · HTML: https://mlsanity.com/paper/2201.11624/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 27 Jan 2022.

## Builds on (most cited first, 19 of 19)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [Multivariate LSTM-FCNs for Time Series Classification](https://mlsanity.com/paper/1801.04503/) (2018)
- [EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding](https://mlsanity.com/paper/1507.08240/) (2015)
- [End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results](https://mlsanity.com/paper/1412.1602/) (2014)
- [Tensor-Train Recurrent Neural Networks for Video Classification](https://mlsanity.com/paper/1707.01786/) (2017)
- [Noisy Activation Functions](https://mlsanity.com/paper/1603.00391/) (2016)
- [Addressing the Rare Word Problem in Neural Machine Translation](https://mlsanity.com/paper/1410.8206/) (2014)
- [Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1610.09975/) (2016)
- [Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation](https://mlsanity.com/paper/1506.07452/) (2015)
- [Gated Word-Character Recurrent Language Model](https://mlsanity.com/paper/1606.01700/) (2016)
- [Reduced-Gate Convolutional LSTM Using Predictive Coding for Spatiotemporal Prediction](https://mlsanity.com/paper/1810.07251/) (2018)

## Built on by (most cited first, 5 of 5)

- [Millimeter Wave Radar-based Human Activity Recognition for Healthcare Monitoring Robot](https://mlsanity.com/paper/2405.01882/) (2024)
- [Asynchronous and Segmented Bidirectional Encoding for NMT](https://mlsanity.com/paper/2402.14849/) (2024)
- [Deep Learning Approaches for Detecting Adversarial Cyberbullying and Hate Speech in Social Networks](https://mlsanity.com/paper/2406.17793/) (2024)
- [LiteLSTM Architecture Based on Weights Sharing for Recurrent Neural Networks](https://mlsanity.com/paper/2301.04794/) (2023)
- [Targeted Adversarial Traffic Generation : Black-box Approach to Evade Intrusion Detection Systems in IoT Networks](https://mlsanity.com/paper/2603.23438/) (2026)

## Abstract (arXiv)

Long short-term memory (LSTM) is a robust recurrent neural network architecture for learning spatiotemporal sequential data. However, it requires significant computational power for learning and implementing from both software and hardware aspects. This paper proposes a novel LiteLSTM architecture based on reducing the computation components of the LSTM using the weights sharing concept to reduce the overall architecture cost and maintain the architecture performance. The proposed LiteLSTM can be significant for learning big data where time-consumption is crucial such as the security of IoT devices and medical data. Moreover, it helps to reduce the CO2 footprint. The proposed model was evaluated and tested empirically on two different datasets from computer vision and cybersecurity domains.