# Task-Aware Information Routing from Common Representation Space in Lifelong Learning

arXiv: https://arxiv.org/abs/2302.11346 · HTML: https://mlsanity.com/paper/2302.11346/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 55 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Co2L: Contrastive Continual Learning](https://mlsanity.com/paper/2106.14413/) (2021)
- [New Insights on Reducing Abrupt Representation Change in Online Continual Learning](https://mlsanity.com/paper/2104.05025/) (2021)
- [Compacting, Picking and Growing for Unforgetting Continual Learning](https://mlsanity.com/paper/1910.06562/) (2019)
- [Deep Learning Through the Lens of Example Difficulty](https://mlsanity.com/paper/2106.09647/) (2021)
- [Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System](https://mlsanity.com/paper/2201.12604/) (2022)
- [Continuous Learning in Single-Incremental-Task Scenarios](https://mlsanity.com/paper/1806.08568/) (2018)
- [The Consciousness Prior](https://mlsanity.com/paper/1709.08568/) (2017)
- [Continual Learning via Neural Pruning](https://mlsanity.com/paper/1903.04476/) (2019)
- [Measuring and regularizing networks in function space](https://mlsanity.com/paper/1805.08289/) (2018)
- [Multivariate Confidence Calibration for Object Detection](https://mlsanity.com/paper/2004.13546/) (2020)
- [Efficient Continual Learning with Modular Networks and Task-Driven Priors](https://mlsanity.com/paper/2012.12631/) (2020)
- [On the geometry of generalization and memorization in deep neural networks](https://mlsanity.com/paper/2105.14602/) (2021)
- [Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks](https://mlsanity.com/paper/1702.05870/) (2017)
- [NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks](https://mlsanity.com/paper/2206.09117/) (2022)
- [Lifelong Learning of Compositional Structures](https://mlsanity.com/paper/2007.07732/) (2020)
- [On the link between conscious function and general intelligence in humans and machines](https://mlsanity.com/paper/2204.05133/) (2022)
- [Consistency is the key to further mitigating catastrophic forgetting in continual learning](https://mlsanity.com/paper/2207.04998/) (2022)
- [Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach](https://mlsanity.com/paper/2207.06267/) (2022)
- [Self-Supervised Learning Aided Class-Incremental Lifelong Learning](https://mlsanity.com/paper/2006.05882/) (2020)
- [Biologically inspired sleep algorithm for artificial neural networks](https://mlsanity.com/paper/1908.02240/) (2019)
- [Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation](https://mlsanity.com/paper/1912.09091/) (2019)

## Built on by (most cited first, 22 of 22)

- [FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning](https://mlsanity.com/paper/2309.14062/) (2023)
- [Make Continual Learning Stronger via C-Flat](https://mlsanity.com/paper/2404.00986/) (2024)
- [BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning](https://mlsanity.com/paper/2305.04769/) (2023)
- [Continual Learning with Strong Experience Replay](https://mlsanity.com/paper/2305.13622/) (2023)
- [Beyond Unimodal Learning: The Importance of Integrating Multiple Modalities for Lifelong Learning](https://mlsanity.com/paper/2405.02766/) (2024)
- [New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning](https://mlsanity.com/paper/2302.08243/) (2023)
- [Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaboration](https://mlsanity.com/paper/2403.11373/) (2024)
- [Lifelong Learning with Task-Specific Adaptation: Addressing the Stability-Plasticity Dilemma](https://mlsanity.com/paper/2503.06213/) (2025)
- [Parameter Efficient Continual Learning with Dynamic Low-Rank Adaptation](https://mlsanity.com/paper/2505.11998/) (2025)
- [Look-Ahead Selective Plasticity for Continual Learning of Visual Tasks](https://mlsanity.com/paper/2311.01617/) (2023)
- [Mitigating Interference in the Knowledge Continuum through Attention-Guided Incremental Learning](https://mlsanity.com/paper/2405.13978/) (2024)
- [Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning](https://mlsanity.com/paper/2403.06535/) (2024)
- [C-Flat++: Towards a More Efficient and Powerful Framework for Continual Learning](https://mlsanity.com/paper/2508.18860/) (2025)
- [A Faster Path to Continual Learning](https://mlsanity.com/paper/2604.11064/) (2026)
- [Prior-Free Continual Learning with Unlabeled Data in the Wild](https://mlsanity.com/paper/2310.10417/) (2023)
- [Retrospective Feature Estimation for Continual Learning](https://mlsanity.com/paper/2406.17381/) (2024)
- [Towards Robust Incremental Learning under Ambiguous Supervision](https://mlsanity.com/paper/2501.13584/) (2025)
- [Continual Learning Beyond Experience Rehearsal and Full Model Surrogates](https://mlsanity.com/paper/2505.21942/) (2025)
- [Global Pre-fixing, Local Adjusting: A Simple yet Effective Contrastive Strategy for Continual Learning](https://mlsanity.com/paper/2509.15347/) (2025)
- [HAD: Heterogeneity-Aware Distillation for Lifelong Heterogeneous Learning](https://mlsanity.com/paper/2603.26192/) (2026)
- [Modular Continual Learning via Zero-Leakage Reconstruction Routing and Autonomous Task Discovery](https://mlsanity.com/paper/2604.14375/) (2026)
- [FreeMOCA: Memory-Free Continual Learning for Malicious Code Analysis](https://mlsanity.com/paper/2605.09664/) (2026)

## Abstract (arXiv)

Intelligent systems deployed in the real world suffer from catastrophic forgetting when exposed to a sequence of tasks. Humans, on the other hand, acquire, consolidate, and transfer knowledge between tasks that rarely interfere with the consolidated knowledge. Accompanied by self-regulated neurogenesis, continual learning in the brain is governed by a rich set of neurophysiological processes that harbor different types of knowledge, which are then integrated by conscious processing. Thus, inspired by the Global Workspace Theory of conscious information access in the brain, we propose TAMiL, a continual learning method that entails task-attention modules to capture task-specific information from the common representation space. We employ simple, undercomplete autoencoders to create a communication bottleneck between the common representation space and the global workspace, allowing only the task-relevant information to the global workspace, thus greatly reducing task interference. Experimental results show that our method outperforms state-of-the-art rehearsal-based and dynamic sparse approaches and bridges the gap between fixed capacity and parameter isolation approaches while being scalable. We also show that our method effectively mitigates catastrophic forgetting while being well-calibrated with reduced task-recency bias.