mlsanityData to 13 Oct 2026

Class-incremental Learning via Deep Model Consolidation

arXiv 1903.07864 ↗First posted 19 Mar 2019cs.CV cs.LGcited by 172

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) often suffer from "catastrophic forgetting" during incremental learning (IL) --- an abrupt degradation of performance on the original set of classes when the training objective is adapted to a newly added set of classes. Existing IL approaches tend to produce a model that is biased towards either the old classes or new classes, unless with the help of exemplars of the old data. To address this issue, we propose a class-incremental learning paradigm called Deep Model Consolidation (DMC), which works well even when the original training data is not available. The idea is to first train a separate model only for the new classes, and then combine the two individual models trained on data of two distinct set of classes (old classes and new classes) via a novel double distillation training objective. The two existing models are consolidated by exploiting publicly available unlabeled auxiliary data. This overcomes the potential difficulties due to the unavailability of original training data. Compared to the state-of-the-art techniques, DMC demonstrates significantly better performance in image classification (CIFAR-100 and CUB-200) and object detection (PASCAL VOC 2007) in the single-headed IL setting.

Key facts. Cited by 172 papers in the map (data through 13 Oct 2026). First posted on arXiv 19 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 23; 2022: 39; 2023: 54; 2024: 34; 2025: 16; 2026: 6.

172 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 862
  2. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 245
  3. 2023Class-Incremental Learning: A Surveycited by 209
  4. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  5. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 116
  6. 2021Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learningcited by 115
  7. 2023TARGET: Federated Class-Continual Learning via Exemplar-Free Distillationcited by 65
  8. 2022ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protectioncited by 57
  9. 2024InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructionscited by 57
  10. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  11. 2021Self-Supervised Training Enhances Online Continual Learningcited by 54
  12. 2022An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditionscited by 54
38 more
  1. 2024Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learningcited by 53
  2. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  3. 2021Efficient Feature Transformations for Discriminative and Generative Continual Learningcited by 50
  4. 2024Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusioncited by 47
  5. 2022Memory Efficient Continual Learning with Transformerscited by 43
  6. 2023Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learningcited by 41
  7. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41
  8. 2022Multi-domain Learning for Updating Face Anti-spoofing Modelscited by 39
  9. 2023Parameter-Level Soft-Masking for Continual Learningcited by 39
  10. 2021Multi-View Correlation Distillation for Incremental Object Detectioncited by 36
  11. 2024Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Modelscited by 34
  12. 2022Sample Condensation in Online Continual Learningcited by 31
  13. 2023Class-Incremental Learning using Diffusion Model for Distillation and Replaycited by 31
  14. 2022Balancing Stability and Plasticity through Advanced Null Space in Continual Learningcited by 30
  15. 2022Meta-attention for ViT-backed Continual Learningcited by 29
  16. 2022Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentationcited by 28
  17. 2021Continual Semi-Supervised Learning through Contrastive Interpolation Consistencycited by 25
  18. 2022Continual Object Detection: A review of definitions, strategies, and challengescited by 25
  19. 2022SATS: Self-Attention Transfer for Continual Semantic Segmentationcited by 24
  20. 2022Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localizationcited by 24
  21. 2023Dealing with Cross-Task Class Discrimination in Online Continual Learningcited by 24
  22. 2023FedET: A Communication-Efficient Federated Class-Incremental Learning Framework Based on Enhanced Transformercited by 23
  23. 2024VLM-PL: Advanced Pseudo Labeling Approach for Class Incremental Object Detection via Vision-Language Modelcited by 23
  24. 2023Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospectscited by 22
  25. 2022Learning to Imagine: Diversify Memory for Incremental Learning using Unlabeled Datacited by 19
  26. 2023Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Modelcited by 19
  27. 2021On the importance of cross-task features for class-incremental learningcited by 18
  28. 2023CoinSeg: Contrast Inter- and Intra- Class Representations for Incremental Segmentationcited by 18
  29. 2022Continual Learning for Visual Search with Backward Consistent Feature Embeddingcited by 17
  30. 2022Continual Learning with Transformers for Image Classificationcited by 17
  31. 2022Continual Prune-and-Select: Class-incremental learning with specialized subnetworkscited by 17
  32. 2023Continual Graph Learning: A Surveycited by 17
  33. 2023ConPET: Continual Parameter-Efficient Tuning for Large Language Modelscited by 17
  34. 2025A Survey of Continual Reinforcement Learningcited by 17
  35. 2022SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledgecited by 16
  36. 2023Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learningcited by 16
  37. 2022Hierarchical Memory Learning for Fine-Grained Scene Graph Generationcited by 15
  38. 2021Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Networkcited by 14

50 most cited of 172 All 172 (JSON)