# In Two Minds about Lifelong Learning: Exploring Hemispheric Redundancy and Specialisation in Neural Models

arXiv: https://arxiv.org/abs/2608.19514 · HTML: https://mlsanity.com/paper/2608.19514/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 57 references that are in the map.
- First posted on arXiv 20 Aug 2026.

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [Online Continual Learning with Maximally Interfered Retrieval](https://mlsanity.com/paper/1908.04742/) (2019)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Understanding the Role of Training Regimes in Continual Learning](https://mlsanity.com/paper/2006.06958/) (2020)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Generative Feature Replay For Class-Incremental Learning](https://mlsanity.com/paper/2004.09199/) (2020)
- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [SpaceNet: Make Free Space For Continual Learning](https://mlsanity.com/paper/2007.07617/) (2020)
- [The Effectiveness of Memory Replay in Large Scale Continual Learning](https://mlsanity.com/paper/2010.02418/) (2020)
- [Load Balancing Mixture of Experts with Similarity Preserving Routers](https://mlsanity.com/paper/2506.14038/) (2025)
- [Is forgetting less a good inductive bias for forward transfer?](https://mlsanity.com/paper/2303.08207/) (2023)
- [Knowledge transfer in deep block-modular neural networks](https://mlsanity.com/paper/1908.08017/) (2019)
- [AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning](https://mlsanity.com/paper/2308.03810/) (2023)
- [Joint Diffusion models in Continual Learning](https://mlsanity.com/paper/2411.08224/) (2024)
- [Learning without Isolation: Pathway Protection for Continual Learning](https://mlsanity.com/paper/2505.18568/) (2025)
- [Looking through the past: better knowledge retention for generative replay in continual learning](https://mlsanity.com/paper/2309.10012/) (2023)

## Abstract (arXiv)

Persistent intelligent systems require the ability to learn continually, but current machine learning approaches face significant challenges in this area compared to biological learning systems. Machine learning algorithms typically trade off retention of previously learned information and adaptation to new or changing data patterns. When continual learning capabilities are absent, algorithms must undergo retraining using the entire data set, an approach that becomes impractical when original training data are unavailable due to storage constraints, financial or computational costs, or privacy restrictions. However, biological animals can learn continually, without experiencing catastrophic forgetting. This paper attempts to build a high-level framework for how animals learn and preserve knowledge by modelling neural components and states that are known to be related to memory consolidation. We focus on three concepts: experience replay, REM sleep, and bilaterality. We propose 4MAS (4 Module Awake/Sleep), a novel macroarchitecture demonstrating how machine learning models might benefit from asymmetric hemispheres, each with their own long- and short-term memory mechanisms, and how a period of sleep between incremental learning tasks might benefit memory consolidation. Finally, we present results showing that our architecture achieves statistical improvements over established generative replay baselines on Split-MNIST (98.3%) and Split-Fashion-MNIST (84.9%), while reducing forgetting by more than half relative to the strongest replay baseline and demonstrating monotonic capacity scaling on Split-CIFAR-100 (29.29%).