# Continual Learning with Knowledge Transfer for Sentiment Classification

arXiv: https://arxiv.org/abs/2112.10021 · HTML: https://mlsanity.com/paper/2112.10021/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 18 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Overcoming catastrophic forgetting with hard attention to the task](https://mlsanity.com/paper/1801.01423/) (2018)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [Overcoming Catastrophic Forgetting by Incremental Moment Matching](https://mlsanity.com/paper/1703.08475/) (2017)
- [Continual Learning of Context-dependent Processing in Neural Networks](https://mlsanity.com/paper/1810.01256/) (2018)
- [A continual learning survey: Defying forgetting in classification tasks](https://mlsanity.com/paper/1909.08383/) (2019)
- [Deep Generative Dual Memory Network for Continual Learning](https://mlsanity.com/paper/1710.10368/) (2017)
- [Continual Learning in Generative Adversarial Nets](https://mlsanity.com/paper/1705.08395/) (2017)
- [Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay](https://mlsanity.com/paper/1903.04566/) (2019)
- [Lifelong Learning for Sentiment Classification](https://mlsanity.com/paper/1801.02808/) (2018)
- [Lifelong Learning CRF for Supervised Aspect Extraction](https://mlsanity.com/paper/1705.00251/) (2017)
- [Forward and Backward Knowledge Transfer for Sentiment Classification](https://mlsanity.com/paper/1906.03506/) (2019)

## Built on by (most cited first, 12 of 12)

- [Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations](https://mlsanity.com/paper/2306.04618/) (2023)
- [Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time](https://mlsanity.com/paper/2211.14238/) (2022)
- [Continual Learning of Natural Language Processing Tasks: A Survey](https://mlsanity.com/paper/2211.12701/) (2022)
- [Memory Efficient Continual Learning with Transformers](https://mlsanity.com/paper/2203.04640/) (2022)
- [Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy](https://mlsanity.com/paper/2312.10549/) (2023)
- [Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2405.05496/) (2024)
- [ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks](https://mlsanity.com/paper/2405.14211/) (2024)
- [CLoG: Benchmarking Continual Learning of Image Generation Models](https://mlsanity.com/paper/2406.04584/) (2024)
- [HOP to the Next Tasks and Domains for Continual Learning in NLP](https://mlsanity.com/paper/2402.18449/) (2024)
- [Context selectivity with dynamic availability enables lifelong continual learning](https://mlsanity.com/paper/2306.01690/) (2023)
- [OpenCML: End-to-End Framework of Open-world Machine Learning to Learn Unknown Classes Incrementally](https://mlsanity.com/paper/2511.19491/) (2025)
- [Relative Parameter Importance in Task-Agnostic Replay-Free Continual Learning](https://mlsanity.com/paper/2608.00630/) (2026)

## Abstract (arXiv)

This paper studies continual learning (CL) for sentiment classification (SC). In this setting, the CL system learns a sequence of SC tasks incrementally in a neural network, where each task builds a classifier to classify the sentiment of reviews of a particular product category or domain. Two natural questions are: Can the system transfer the knowledge learned in the past from the previous tasks to the new task to help it learn a better model for the new task? And, can old models for previous tasks be improved in the process as well? This paper proposes a novel technique called KAN to achieve these objectives. KAN can markedly improve the SC accuracy of both the new task and the old tasks via forward and backward knowledge transfer. The effectiveness of KAN is demonstrated through extensive experiments.