# Investigating Forgetting in Pre-Trained Representations Through Continual Learning

arXiv: https://arxiv.org/abs/2305.05968 · HTML: https://mlsanity.com/paper/2305.05968/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 40 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [SentEval: An Evaluation Toolkit for Universal Sentence Representations](https://mlsanity.com/paper/1803.05449/) (2018)
- [Semantic Drift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2004.00440/) (2020)
- [Episodic Memory in Lifelong Language Learning](https://mlsanity.com/paper/1906.01076/) (2019)
- [What Happens To BERT Embeddings During Fine-tuning?](https://mlsanity.com/paper/2004.14448/) (2020)
- [Continual Learning for Text Classification with Information Disentanglement Based Regularization](https://mlsanity.com/paper/2104.05489/) (2021)
- [Continual Learning of Natural Language Processing Tasks: A Survey](https://mlsanity.com/paper/2211.12701/) (2022)

## Built on by (most cited first, 21 of 21)

- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Orthogonal Subspace Learning for Language Model Continual Learning](https://mlsanity.com/paper/2310.14152/) (2023)
- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs](https://mlsanity.com/paper/2407.15549/) (2024)
- [Eight Methods to Evaluate Robust Unlearning in LLMs](https://mlsanity.com/paper/2402.16835/) (2024)
- [Defending Against Unforeseen Failure Modes with Latent Adversarial Training](https://mlsanity.com/paper/2403.05030/) (2024)
- [Investigating Continual Pretraining in Large Language Models: Insights and Implications](https://mlsanity.com/paper/2402.17400/) (2024)
- [Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic](https://mlsanity.com/paper/2408.13656/) (2024)
- [SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models](https://mlsanity.com/paper/2401.08295/) (2024)
- [MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector](https://mlsanity.com/paper/2408.08661/) (2024)
- [PMoE: Progressive Mixture of Experts with Asymmetric Transformer for Continual Learning](https://mlsanity.com/paper/2407.21571/) (2024)
- [DiTASK: Multi-Task Fine-Tuning with Diffeomorphic Transformations](https://mlsanity.com/paper/2502.06029/) (2025)
- [Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective](https://mlsanity.com/paper/2505.20970/) (2025)
- [Balancing Speciality and Versatility: A Coarse to Fine Framework for Mitigating Catastrophic Forgetting in Large Language Models](https://mlsanity.com/paper/2404.10306/) (2024)
- [When Life gives you LLMs, make LLM-ADE: Large Language Models with Adaptive Data Engineering](https://mlsanity.com/paper/2404.13028/) (2024)
- [Efficient Continual Learning for Small Language Models with a Discrete Key-Value Bottleneck](https://mlsanity.com/paper/2412.08528/) (2024)
- [On Understanding of the Dynamics of Model Capacity in Continual Learning](https://mlsanity.com/paper/2508.08052/) (2025)
- [The Persistent Vulnerability of Aligned AI Systems](https://mlsanity.com/paper/2604.00324/) (2026)
- [PEFT-MedSAM: Efficient Fine-Tuning of Medical Foundation Models for Explainable Skin Lesion Segmentation](https://mlsanity.com/paper/2606.18707/) (2026)
- [Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning](https://mlsanity.com/paper/2607.23837/) (2026)
- [Task Vector Descent: Learning from Non-IID Batches](https://mlsanity.com/paper/2610.05402/) (2026)

## Abstract (arXiv)

Representation forgetting refers to the drift of contextualized representations during continual training. Intuitively, the representation forgetting can influence the general knowledge stored in pre-trained language models (LMs), but the concrete effect is still unclear. In this paper, we study the effect of representation forgetting on the generality of pre-trained language models, i.e. the potential capability for tackling future downstream tasks. Specifically, we design three metrics, including overall generality destruction (GD), syntactic knowledge forgetting (SynF), and semantic knowledge forgetting (SemF), to measure the evolution of general knowledge in continual learning. With extensive experiments, we find that the generality is destructed in various pre-trained LMs, and syntactic and semantic knowledge is forgotten through continual learning. Based on our experiments and analysis, we further get two insights into alleviating general knowledge forgetting: 1) training on general linguistic tasks at first can mitigate general knowledge forgetting; 2) the hybrid continual learning method can mitigate the generality destruction and maintain more general knowledge compared with those only considering rehearsal or regularization.