mlsanityData to 13 Oct 2026

Continual Learning via Inter-Task Synaptic Mapping

arXiv 2106.13954 ↗First posted 26 Jun 2021cs.LG cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Learning from streaming tasks leads a model to catastrophically erase unique experiences it absorbs from previous episodes. While regularization techniques such as LWF, SI, EWC have proven themselves as an effective avenue to overcome this issue by constraining important parameters of old tasks from changing when accepting new concepts, these approaches do not exploit common information of each task which can be shared to existing neurons. As a result, they do not scale well to large-scale problems since the parameter importance variables quickly explode. An Inter-Task Synaptic Mapping (ISYANA) is proposed here to underpin knowledge retention for continual learning. ISYANA combines task-to-neuron relationship as well as concept-to-concept relationship such that it prevents a neuron to embrace distinct concepts while merely accepting relevant concept. Numerical study in the benchmark continual learning problems has been carried out followed by comparison against prominent continual learning algorithms. ISYANA exhibits competitive performance compared to state of the arts. Codes of ISYANA is made available in urlhttps://github.com/ContinualAL/ISYANAKBS.

Key facts. Cited by 14 papers in the map (data through 13 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 26 Jun 2021.

Builds on

20 of 27 references.

References · 20

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,753
  2. 2016Learning without Forgettingcited by 2,479
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,323
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,882
  5. 2015Spatial Transformer Networkscited by 1,425
  6. 2016Progressive Neural Networkscited by 1,347
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,322
  8. 2017Continual Learning with Deep Generative Replaycited by 1,155
  9. 2018Efficient Lifelong Learning with A-GEMcited by 1,053
  10. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 718
  11. 2017Lifelong Learning with Dynamically Expandable Networkscited by 692
  12. 2019Three scenarios for continual learningcited by 612
8 more
  1. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  2. 2019Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgettingcited by 302
  3. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  4. 2016Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clusteringcited by 237
  5. 2018Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilizationcited by 104
  6. 2018Generative replay with feedback connections as a general strategy for continual learningcited by 104
  7. 2019Overcoming Catastrophic Forgetting by Neuron-level Plasticity Controlcited by 36
  8. 2019Automatic Construction of Multi-layer Perceptron Network from Streaming Examplescited by 11

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 3; 2024: 4; 2025: 3; 2026: 1.

14 papers.

Sort:

By area:

  1. 2022Reinforced Continual Learning for Graphscited by 24
  2. 2021Unsupervised Continual Learning via Self-Adaptive Deep Clustering Approachcited by 19
  3. 2025A Survey of Continual Reinforcement Learningcited by 17
  4. 2023AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learningcited by 9
  5. 2023Assessor-Guided Learning for Continual Environmentscited by 7
  6. 2022Scalable Adversarial Online Continual Learningcited by 6
  7. 2023Few-Shot Continual Learning via Flat-to-Wide Approachescited by 4
  8. 2024Few-Shot Class Incremental Learning via Robust Transformer Approachcited by 4
  9. 2025Continual Knowledge Consolidation LORA for Domain Incremental Learningcited by 2
  10. 2024Cross-Domain Continual Learning via CLAMPcited by 1
  11. 2024Unsupervised Few-Shot Continual Learning for Remote Sensing Image Scene Classificationcited by 1
  12. 2025Information-Theoretic Dual Memory System for Continual Learningcited by 1
2 more
  1. 2024MGSER-SAM: Memory-Guided Soft Experience Replay with Sharpness-Aware Optimization for Enhanced Continual Learningcited by 0
  2. 2026Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidationcited by 0