# MinimalRNN: Toward More Interpretable and Trainable Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1711.06788 · HTML: https://mlsanity.com/paper/1711.06788/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2017.

## Built on by (most cited first, 10 of 10)

- [Fine-Tuning Pre-trained Transformers into Decaying Fast Weights](https://mlsanity.com/paper/2210.04243/) (2022)
- [Learning Spatio-Temporal Model of Disease Progression with NeuralODEs from Longitudinal Volumetric Data](https://mlsanity.com/paper/2211.04234/) (2022)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [Deep Geometric Learning with Monotonicity Constraints for Alzheimer's Disease Progression](https://mlsanity.com/paper/2310.03353/) (2023)
- [Sparse Convolutional Recurrent Learning for Efficient Event-based Neuromorphic Object Detection](https://mlsanity.com/paper/2506.13440/) (2025)
- [Transformer Model for Alzheimer's Disease Progression Prediction Using Longitudinal Visit Sequences](https://mlsanity.com/paper/2507.03899/) (2025)
- [DeepAD: A Robust Deep Learning Model of Alzheimer's Disease Progression for Real-World Clinical Applications](https://mlsanity.com/paper/2203.09096/) (2022)
- [Context-aware Sparse Spatiotemporal Learning for Event-based Vision](https://mlsanity.com/paper/2508.19806/) (2025)
- [Multi-View Imputation and Cross-Attention Network Based on Incomplete Longitudinal and Multimodal Data for Conversion Prediction of Mild Cognitive Impairment](https://mlsanity.com/paper/2206.08019/) (2022)
- [Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents](https://mlsanity.com/paper/2610.02525/) (2026)

## Abstract (arXiv)

We introduce MinimalRNN, a new recurrent neural network architecture that achieves comparable performance as the popular gated RNNs with a simplified structure. It employs minimal updates within RNN, which not only leads to efficient learning and testing but more importantly better interpretability and trainability. We demonstrate that by endorsing the more restrictive update rule, MinimalRNN learns disentangled RNN states. We further examine the learning dynamics of different RNN structures using input-output Jacobians, and show that MinimalRNN is able to capture longer range dependencies than existing RNN architectures.