mlsanityData to 7 Oct 2026

Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks

arXiv 2112.03271 ↗First posted 6 Dec 2021cs.CL cs.AI cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks. Although some CL techniques have been proposed for document sentiment classification, we are not aware of any CL work on ASC. A CL system that incrementally learns a sequence of ASC tasks should address the following two issues: (1) transfer knowledge learned from previous tasks to the new task to help it learn a better model, and (2) maintain the performance of the models for previous tasks so that they are not forgotten. This paper proposes a novel capsule network based model called B-CL to address these issues. B-CL markedly improves the ASC performance on both the new task and the old tasks via forward and backward knowledge transfer. The effectiveness of B-CL is demonstrated through extensive experiments.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 32 of its 51 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

32 of 51 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  8. 2017Continual Learning with Deep Generative Replaycited by 1,152
  9. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  10. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  11. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  12. 2017Dynamic Routing Between Capsulescited by 689
20 more
  1. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  2. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  3. 2018Learning Word Vectors for 157 Languagescited by 279
  4. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  5. 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 178
  6. 2019BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysiscited by 176
  7. 2019Uncertainty-based Continual Learning with Adaptive Regularizationcited by 144
  8. 2019Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentencecited by 121
  9. 2019A continual learning survey: Defying forgetting in classification taskscited by 118
  10. 2020What Happens To BERT Embeddings During Fine-tuning?cited by 113
  11. 2016Aspect Level Sentiment Classification with Deep Memory Networkcited by 82
  12. 2018Aspect Based Sentiment Analysis with Gated Convolutional Networkscited by 65
  13. 2017Deep Generative Dual Memory Network for Continual Learningcited by 59
  14. 2017Continual Learning in Generative Adversarial Netscited by 51
  15. 2019Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replaycited by 51
  16. 2018Lifelong Learning for Sentiment Classificationcited by 21
  17. 2019Towards Scalable and Reliable Capsule Networks for Challenging NLP Applicationscited by 10
  18. 2017Lifelong Learning CRF for Supervised Aspect Extractioncited by 9
  19. 2019Forward and Backward Knowledge Transfer for Sentiment Classificationcited by 9
  20. 2017Dual Attention Network for Product Compatibility and Function Satisfiability Analysiscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 17; 2023: 15; 2024: 10; 2025: 3; 2026: 1.

46 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2023Continual Pre-training of Language Modelscited by 194
  3. 2022Fine-tuned Language Models are Continual Learnerscited by 152
  4. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  5. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 85
  6. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  7. 2023Modular Deep Learningcited by 82
  8. 2022A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challengescited by 61
  9. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 60
  10. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  11. 2022Memory Efficient Continual Learning with Transformerscited by 43
  12. 2022Continual Training of Language Models for Few-Shot Learningcited by 33
34 more
  1. 2022Continual Sequence Generation with Adaptive Compositional Modulescited by 31
  2. 2022Meta-attention for ViT-backed Continual Learningcited by 29
  3. 2022ConTinTin: Continual Learning from Task Instructionscited by 26
  4. 2022Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detectioncited by 23
  5. 2024Unlocking Continual Learning Abilities in Language Modelscited by 22
  6. 2023Class Incremental Learning via Likelihood Ratio Based Task Predictioncited by 17
  7. 2023Carpe Diem: On the Evaluation of World Knowledge in Lifelong Language Modelscited by 17
  8. 2022Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuningcited by 12
  9. 2024ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Taskscited by 11
  10. 2024Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysiscited by 10
  11. 2022Parameter-Efficient Finetuning for Robust Continual Multilingual Learningcited by 9
  12. 2023Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extractioncited by 9
  13. 2023Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Promptcited by 8
  14. 2023Sub-network Discovery and Soft-masking for Continual Learning of Mixed Taskscited by 7
  15. 2023Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Modelscited by 7
  16. 2024In-context Continual Learning Assisted by an External Continual Learnercited by 7
  17. 2022Feature Forgetting in Continual Representation Learningcited by 6
  18. 2023Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognitioncited by 5
  19. 2023Emotion Prediction Oriented method with Multiple Supervisions for Emotion-Cause Pair Extractioncited by 4
  20. 2023Visually Grounded Continual Language Learning with Selective Specializationcited by 4
  21. 2025Task-Core Memory Management and Consolidation for Long-term Continual Learningcited by 4
  22. 2025Self-Evolving LLMs via Continual Instruction Tuningcited by 4
  23. 2024HOP to the Next Tasks and Domains for Continual Learning in NLPcited by 3
  24. 2024Mitigate Negative Transfer with Similarity Heuristic Lifelong Prompt Tuningcited by 3
  25. 2024Learn it or Leave it: Module Composition and Pruning for Continual Learningcited by 3
  26. 2022Unified Question Generation with Continual Lifelong Learningcited by 2
  27. 2022Domain-Aware Contrastive Knowledge Transfer for Multi-domain Imbalanced Datacited by 2
  28. 2025DoMIX: An Efficient Framework for Exploiting Domain Knowledge in Fine-Tuningcited by 2
  29. 2022Variational Autoencoder with Disentanglement Priors for Low-Resource Task-Specific Natural Language Generationcited by 1
  30. 2023Improving Representational Continuity via Continued Pretrainingcited by 1
  31. 2023Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Surveycited by 1
  32. 2024EdgeOL: Efficient in-situ Online Learning on Edge Devicescited by 1
  33. 2023Context selectivity with dynamic availability enables lifelong continual learningcited by 0
  34. 2026Rethinking Transfer in Continual Learning: A Replay-Based Realisationcited by 0