# Utilizing Priming to Identify Optimal Class Ordering to Alleviate Catastrophic Forgetting

arXiv: https://arxiv.org/abs/2212.12643 · HTML: https://mlsanity.com/paper/2212.12643/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 24 Dec 2022.

## Builds on (most cited first, 16 of 16)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Overcoming Catastrophic Forgetting by Incremental Moment Matching](https://mlsanity.com/paper/1703.08475/) (2017)
- [REMIND Your Neural Network to Prevent Catastrophic Forgetting](https://mlsanity.com/paper/1910.02509/) (2019)
- [Visual Semantic Reasoning for Image-Text Matching](https://mlsanity.com/paper/1909.02701/) (2019)
- [Reducing Network Agnostophobia](https://mlsanity.com/paper/1811.04110/) (2018)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Leveraging Seen and Unseen Semantic Relationships for Generative Zero-Shot Learning](https://mlsanity.com/paper/2007.09549/) (2020)
- [Psycholinguistics meets Continual Learning: Measuring Catastrophic Forgetting in Visual Question Answering](https://mlsanity.com/paper/1906.04229/) (2019)
- [Visual Semantic Information Pursuit: A Survey](https://mlsanity.com/paper/1903.05434/) (2019)
- [On Class Orderings for Incremental Learning](https://mlsanity.com/paper/2007.02145/) (2020)
- [Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers](https://mlsanity.com/paper/2109.05186/) (2021)
- [Incremental Deep Neural Network Learning using Classification Confidence Thresholding](https://mlsanity.com/paper/2106.11437/) (2021)

## Abstract (arXiv)

In order for artificial neural networks to begin accurately mimicking biological ones, they must be able to adapt to new exigencies without forgetting what they have learned from previous training. Lifelong learning approaches to artificial neural networks attempt to strive towards this goal, yet have not progressed far enough to be realistically deployed for natural language processing tasks. The proverbial roadblock of catastrophic forgetting still gate-keeps researchers from an adequate lifelong learning model. While efforts are being made to quell catastrophic forgetting, there is a lack of research that looks into the importance of class ordering when training on new classes for incremental learning. This is surprising as the ordering of "classes" that humans learn is heavily monitored and incredibly important. While heuristics to develop an ideal class order have been researched, this paper examines class ordering as it relates to priming as a scheme for incremental class learning. By examining the connections between various methods of priming found in humans and how those are mimicked yet remain unexplained in life-long machine learning, this paper provides a better understanding of the similarities between our biological systems and the synthetic systems while simultaneously improving current practices to combat catastrophic forgetting. Through the merging of psychological priming practices with class ordering, this paper is able to identify a generalizable method for class ordering in NLP incremental learning tasks that consistently outperforms random class ordering.