Modular Continual Learning via Zero-Leakage Reconstruction Routing and Autonomous Task Discovery
Not on the map. Why?
Abstract (arXiv)
Catastrophic forgetting remains a primary hurdle in sequential task learning for artificial neural networks. We propose a silicon-native modular architecture that achieves structural parameter isolation using Task-Specific Experts and a distributed, outlier-based Gatekeeper. Moving beyond traditional sequential consolidation, our framework utilizes a Simultaneous Pipeline where Teacher learning, Student distillation, and Router manifold acquisition occur in parallel while raw data is present in a localized training session. This approach ensures computational efficiency and complies with privacy mandates like GDPR by deleting raw data as soon as a task is learned. We demonstrate that a Tight-Bottleneck Autoencoder (TB-AE) can effectively distinguish semantically crowded manifolds in high-dimensional latent spaces, overcoming the posterior collapse inherent to standard variational methods. By establishing strict topological boundaries, our TB-AE resolves latent space crowding in 4096-D LLM embeddings to provide a robust, unsupervised novelty signal. Furthermore, we validate an Autonomous Retrieval mechanism that confidently identifies returning manifolds, enabling stable lifelong learning without redundant module instantiation. Empirical results demonstrate that our ``Live Distillation'' approach acts as a natural regularizer, achieving strong retention across computer vision and natural language processing domains without suffering a student fidelity gap.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 19 references that are in the map. First posted on arXiv 15 Apr 2026.
Builds on
13 of 19 references.
References · 13
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2016Learning without Forgettingcited by 2,496
- 2017Gradient Episodic Memory for Continual Learningcited by 1,898
- 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
- 2026Self-Distillation Enables Continual Learningcited by 247
- 2023Task-Aware Information Routing from Common Representation Space in Lifelong Learningcited by 22
- 2023Manifold Learning by Mixture Models of VAEs for Inverse Problemscited by 17
- 2026CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansioncited by 17
- 2021Lifelong Mixture of Variational Autoencoderscited by 10
- 2025Continual Reinforcement Learning via Autoencoder-Driven Task and New Environment Recognitioncited by 5
Built on by
Not cited yet.