# Minion Gated Recurrent Unit for Continual Learning

arXiv: https://arxiv.org/abs/2503.06175 · HTML: https://mlsanity.com/paper/2503.06175/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 8 Mar 2025.

## Builds on (most cited first, 26 of 26)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks](https://mlsanity.com/paper/1701.05923/) (2017)
- [Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting](https://mlsanity.com/paper/1802.02950/) (2018)
- [A Closer Look at Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2203.17269/) (2022)
- [Light Gated Recurrent Units for Speech Recognition](https://mlsanity.com/paper/1803.10225/) (2018)
- [Minimal Gated Unit for Recurrent Neural Networks](https://mlsanity.com/paper/1603.09420/) (2016)
- [Continual Learning for Recurrent Neural Networks: an Empirical Evaluation](https://mlsanity.com/paper/2103.07492/) (2021)
- [A comparison of LSTM and GRU networks for learning symbolic sequences](https://mlsanity.com/paper/2107.02248/) (2021)
- [Simplified Minimal Gated Unit Variations for Recurrent Neural Networks](https://mlsanity.com/paper/1701.03452/) (2017)
- [Design Principles for Lifelong Learning AI Accelerators](https://mlsanity.com/paper/2310.04467/) (2023)
- [Continual Learning with Strong Experience Replay](https://mlsanity.com/paper/2305.13622/) (2023)
- [Simplified Gating in Long Short-term Memory (LSTM) Recurrent Neural Networks](https://mlsanity.com/paper/1701.03441/) (2017)
- [End-to-End Memristive HTM System for Pattern Recognition and Sequence Prediction](https://mlsanity.com/paper/2006.11958/) (2020)
- [CryptoGRU: Low Latency Privacy-Preserving Text Analysis With GRU](https://mlsanity.com/paper/2010.11796/) (2020)
- [Time-Series Forecasting and Sequence Learning Using Memristor-based Reservoir System](https://mlsanity.com/paper/2405.13347/) (2024)
- [Reservoir Network with Structural Plasticity for Human Activity Recognition](https://mlsanity.com/paper/2503.00393/) (2025)

## Built on by (most cited first, 1 of 1)

- [M2RU: Memristive Minion Recurrent Unit for On-Chip Continual Learning at the Edge](https://mlsanity.com/paper/2512.17299/) (2025)

## Abstract (arXiv)

The increasing demand for continual learning in sequential data processing has led to progressively complex training methodologies and larger recurrent network architectures. Consequently, this has widened the knowledge gap between continual learning with recurrent neural networks (RNNs) and their ability to operate on devices with limited memory and compute. To address this challenge, we investigate the effectiveness of simplifying RNN architectures, particularly gated recurrent unit (GRU), and its impact on both single-task and multitask sequential learning. We propose a new variant of GRU, namely the minion recurrent unit (MiRU). MiRU replaces conventional gating mechanisms with scaling coefficients to regulate dynamic updates of hidden states and historical context, reducing computational costs and memory requirements. Despite its simplified architecture, MiRU maintains performance comparable to the standard GRU while achieving 2.90x faster training and reducing parameter usage by 2.88x, as demonstrated through evaluations on sequential image classification and natural language processing benchmarks. The impact of model simplification on its learning capacity is also investigated by performing continual learning tasks with a rehearsal-based strategy and global inhibition. We find that MiRU demonstrates stable performance in multitask learning even when using only rehearsal, unlike the standard GRU and its variants. These features position MiRU as a promising candidate for edge-device applications.