mlsanityData to 14 Oct 2026

Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learning

arXiv 2409.16391 ↗First posted 24 Sep 2024cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We focus on a relatively unexplored learning paradigm known as em Online Unsupervised Continual Learning (O-UCL), where an agent receives a non-stationary, unlabeled data stream and progressively learns to identify an increasing number of classes. This paradigm is designed to model real-world applications where encountering novelty is the norm, such as exploring a terrain with several unknown and time-varying entities. Unlike prior work in unsupervised, continual, or online learning, O-UCL combines all three areas into a single challenging and realistic learning paradigm. In this setting, agents are frequently evaluated and must aim to maintain the best possible representation at any point of the data stream, rather than at the end of pre-specified offline tasks. The proposed approach, called Patch-based Contrastive learning and Memory Consolidation (PCMC), builds a compositional understanding of data by identifying and clustering patch-level features. Embeddings for these patch-level features are extracted with an encoder trained via patch-based contrastive learning. PCMC incorporates new data into its distribution while avoiding catastrophic forgetting, and it consolidates memory examples during ``sleep" periods. We evaluate PCMC's performance on streams created from the ImageNet and Places365 datasets. Additionally, we explore various versions of the PCMC algorithm and compare its performance against several existing methods and simple baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 54 of its 61 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

54 of 61 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2016Learning without Forgettingcited by 2,496
  10. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  12. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
38 more
  1. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  2. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  3. 2017Continual Learning with Deep Generative Replaycited by 1,158
  4. 2016Colorful Image Colorizationcited by 1,114
  5. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  6. 2018Experience Replay for Continual Learningcited by 1,095
  7. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  8. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  9. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  10. 2019Gradient based sample selection for online continual learningcited by 571
  11. 2016Learning Representations for Automatic Colorizationcited by 251
  12. 2021Co2L: Contrastive Continual Learningcited by 244
  13. 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
  14. 2018Task-Free Continual Learningcited by 221
  15. 2019REMIND Your Neural Network to Prevent Catastrophic Forgettingcited by 197
  16. 2021Self-Supervised Models are Continual Learnerscited by 149
  17. 2019Continual Unsupervised Representation Learningcited by 147
  18. 2020A Neural Dirichlet Process Mixture Model for Task-Free Continual Learningcited by 134
  19. 2017Encoder Based Lifelong Learningcited by 102
  20. 2021Representational Continuity for Unsupervised Continual Learningcited by 95
  21. 2019Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysiscited by 92
  22. 2017Lifelong Generative Modelingcited by 51
  23. 2022Online Continual Learning for Embedded Devicescited by 49
  24. 2020Online Continual Learning under Extreme Memory Constraintscited by 48
  25. 2023Real-Time Evaluation in Online Continual Learning: A New Hopecited by 47
  26. 2022Neural Manifold Clustering and Embeddingcited by 44
  27. 2018Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologiescited by 43
  28. 2020Learning latent representations across multiple data domains using Lifelong VAEGANcited by 35
  29. 2019Unsupervised Progressive Learning and the STAM Architecturecited by 30
  30. 2023SIESTA: Efficient Online Continual Learning with Sleepcited by 24
  31. 2022SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledgecited by 16
  32. 2020Self-Supervised Learning Aided Class-Incremental Lifelong Learningcited by 11
  33. 2021Continual Contrastive Learning for Image Classificationcited by 11
  34. 2023Online Continual Learning on Hierarchical Label Expansioncited by 11
  35. 2021Continual Learning on the Edge with TensorFlow Litecited by 9
  36. 2022Memory Population in Continual Learning via Outlier Eliminationcited by 9
  37. 2021Self-Supervised Class Incremental Learningcited by 8
  38. 2023Improving Replay Sample Selection and Storage for Less Forgetting in Continual Learningcited by 7

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.