mlsanityData to 7 Oct 2026

A Distinct Unsupervised Reference Model From The Environment Helps Continual Learning

arXiv 2301.04506 ↗First posted 11 Jan 2023cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The existing continual learning methods are mainly focused on fully-supervised scenarios and are still not able to take advantage of unlabeled data available in the environment. Some recent works tried to investigate semi-supervised continual learning (SSCL) settings in which the unlabeled data are available, but it is only from the same distribution as the labeled data. This assumption is still not general enough for real-world applications and restricts the utilization of unsupervised data. In this work, we introduce Open-Set Semi-Supervised Continual Learning (OSSCL), a more realistic semi-supervised continual learning setting in which out-of-distribution (OoD) unlabeled samples in the environment are assumed to coexist with the in-distribution ones. Under this configuration, we present a model with two distinct parts: (i) the reference network captures general-purpose and task-agnostic knowledge in the environment by using a broad spectrum of unlabeled samples, (ii) the learner network is designed to learn task-specific representations by exploiting supervised samples. The reference model both provides a pivotal representation space and also segregates unlabeled data to exploit them more efficiently. By performing a diverse range of experiments, we show the superior performance of our model compared with other competitors and prove the effectiveness of each component of the proposed model.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 68 references that are in the map. First posted on arXiv 11 Jan 2023.

Builds on

41 of 68 references.

References · 41

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  5. 2016Learning without Forgettingcited by 2,465
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  8. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  10. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  11. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
29 more
  1. 2017Continual Learning with Deep Generative Replaycited by 1,152
  2. 2016Colorful Image Colorizationcited by 1,113
  3. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  4. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  5. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  6. 2019Gradient based sample selection for online continual learningcited by 568
  7. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  8. 2020Meta Pseudo Labelscited by 372
  9. 2017Variational Continual Learningcited by 335
  10. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  11. 2015Semi-Supervised Learning with Ladder Networkscited by 246
  12. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  13. 2021Co2L: Contrastive Continual Learningcited by 243
  14. 2018Selective Experience Replay for Lifelong Learningcited by 239
  15. 2020Supermasks in Superpositioncited by 223
  16. 2016Adversarially Learned Inferencecited by 216
  17. 2017Incremental Learning of Object Detectors without Catastrophic Forgettingcited by 208
  18. 2020Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semanticscited by 158
  19. 2019Overcoming Catastrophic Forgetting with Unlabeled Data in the Wildcited by 111
  20. 2021GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learningcited by 95
  21. 2015Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networkscited by 94
  22. 2020Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learningcited by 82
  23. 2021ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learningcited by 68
  24. 2017Triple Generative Adversarial Netscited by 59
  25. 2021Mixture Proportion Estimation and PU Learning: A Modern Approachcited by 52
  26. 2021Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learningcited by 40
  27. 2021Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffercited by 30
  28. 2020The Effectiveness of Memory Replay in Large Scale Continual Learningcited by 26
  29. 2021OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Datacited by 22

Built on by

Not cited yet.