mlsanityData to 7 Oct 2026

Continual Learning for Sentence Representations Using Conceptors

arXiv 1904.09187 ↗First posted 18 Apr 2019cs.LG stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

Distributed representations of sentences have become ubiquitous in natural language processing tasks. In this paper, we consider a continual learning scenario for sentence representations: Given a sequence of corpora, we aim to optimize the sentence encoder with respect to the new corpus while maintaining its accuracy on the old corpora. To address this problem, we propose to initialize sentence encoders with the help of corpus-independent features, and then sequentially update sentence encoders using Boolean operations of conceptor matrices to learn corpus-dependent features. We evaluate our approach on semantic textual similarity tasks and show that our proposed sentence encoder can continually learn features from new corpora while retaining its competence on previously encountered corpora.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 9; 2023: 3; 2024: 3; 2025: 1; 2026: 2.

22 papers.

Sort:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  3. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 114
  4. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  5. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  6. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  7. 2022Continual Training of Language Models for Few-Shot Learningcited by 33
  8. 2022Just Rank: Rethinking Evaluation with Word and Sentence Similaritiescited by 20
  9. 2023Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio Detectioncited by 11
  10. 2023Preventing Catastrophic Forgetting in Continual Learning of New Natural Language Taskscited by 10
  11. 2025AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projectioncited by 7
  12. 2022Life-long Learning for Multilingual Neural Machine Translation with Knowledge Distillationcited by 5
10 more
  1. 2022RVAE-LAMOL: Residual Variational Autoencoder to Enhance Lifelong Language Learningcited by 4
  2. 2022Unsupervised Sentence Textual Similarity with Compositional Phrase Semanticscited by 4
  3. 2022Conceptor-Aided Debiasing of Large Language Modelscited by 4
  4. 2021Dynamically Addressing Unseen Rumor via Continual Learningcited by 3
  5. 2024HOP to the Next Tasks and Domains for Continual Learning in NLPcited by 3
  6. 2024Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspectivecited by 2
  7. 2022Continual Learning with Global Alignmentcited by 1
  8. 2026Conceptors for Semantic Steeringcited by 1
  9. 2022Ask Question First for Enhancing Lifelong Language Learningcited by 0
  10. 2026Exploring the relationship between team institutional composition and novelty in academic papers based on fine-grained knowledge entitiescited by 0