# Examining Changes in Internal Representations of Continual Learning Models Through Tensor Decomposition

arXiv: https://arxiv.org/abs/2405.03244 · HTML: https://mlsanity.com/paper/2405.03244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics](https://mlsanity.com/paper/2007.07400/) (2020)
- [Continual Learning with Node-Importance based Adaptive Group Sparse Regularization](https://mlsanity.com/paper/2003.13726/) (2020)
- [Reliability of CKA as a Similarity Measure in Deep Learning](https://mlsanity.com/paper/2210.16156/) (2022)
- [Probing Representation Forgetting in Supervised and Unsupervised Continual Learning](https://mlsanity.com/paper/2203.13381/) (2022)
- [What do Vision Transformers Learn? A Visual Exploration](https://mlsanity.com/paper/2212.06727/) (2022)
- [Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping](https://mlsanity.com/paper/2102.11343/) (2021)
- [Avalanche: A PyTorch Library for Deep Continual Learning](https://mlsanity.com/paper/2302.01766/) (2023)
- [Unsupervised visualization of image datasets using contrastive learning](https://mlsanity.com/paper/2210.09879/) (2022)
- [Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting](https://mlsanity.com/paper/2304.00933/) (2023)
- [Feature Forgetting in Continual Representation Learning](https://mlsanity.com/paper/2205.13359/) (2022)

## Abstract (arXiv)

Continual learning (CL) has spurred the development of several methods aimed at consolidating previous knowledge across sequential learning. Yet, the evaluations of these methods have primarily focused on the final output, such as changes in the accuracy of predicted classes, overlooking the issue of representational forgetting within the model. In this paper, we propose a novel representation-based evaluation framework for CL models. This approach involves gathering internal representations from throughout the continual learning process and formulating three-dimensional tensors. The tensors are formed by stacking representations, such as layer activations, generated from several inputs and model `snapshots', throughout the learning process. By conducting tensor component analysis (TCA), we aim to uncover meaningful patterns about how the internal representations evolve, expecting to highlight the merits or shortcomings of examined CL strategies. We conduct our analyses across different model architectures and importance-based continual learning strategies, with a curated task selection. While the results of our approach mirror the difference in performance of various CL strategies, we found that our methodology did not directly highlight specialized clusters of neurons, nor provide an immediate understanding the evolution of filters. We believe a scaled down version of our approach will provide insight into the benefits and pitfalls of using TCA to study continual learning dynamics.