mlsanityData to 7 Oct 2026

Deep Class Incremental Learning from Decentralized Data

arXiv 2203.05984 ↗First posted 11 Mar 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we focus on a new and challenging decentralized machine learning paradigm in which there are continuous inflows of data to be addressed and the data are stored in multiple repositories. We initiate the study of data decentralized class-incremental learning (DCIL) by making the following contributions. Firstly, we formulate the DCIL problem and develop the experimental protocol. Secondly, we introduce a paradigm to create a basic decentralized counterpart of typical (centralized) class-incremental learning approaches, and as a result, establish a benchmark for the DCIL study. Thirdly, we further propose a Decentralized Composite knowledge Incremental Distillation framework (DCID) to transfer knowledge from historical models and multiple local sites to the general model continually. DCID consists of three main components namely local class-incremental learning, collaborated knowledge distillation among local models, and aggregated knowledge distillation from local models to the general one. We comprehensively investigate our DCID framework by using different implementations of the three components. Extensive experimental results demonstrate the effectiveness of our DCID framework. The codes of the baseline methods and the proposed DCIL will be released at https://github.com/zxxxxh/DCIL.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 46 of its 82 references that are in the map. First posted on arXiv 11 Mar 2022.

Builds on

46 of 82 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2016Learning without Forgettingcited by 2,465
  10. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  11. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  12. 2017Continual Learning with Deep Generative Replaycited by 1,152
34 more
  1. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  2. 2018Federated Learning with Non-IID Datacited by 1,099
  3. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  4. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  6. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  7. 2019Large Scale Incremental Learningcited by 727
  8. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  10. 2018End-to-End Incremental Learningcited by 596
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2018Born Again Neural Networkscited by 539
  13. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  14. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  15. 2019Maintaining Discrimination and Fairness in Class Incremental Learningcited by 281
  16. 2020Few-Shot Class-Incremental Learningcited by 249
  17. 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
  18. 2020Mnemonics Training: Multi-Class Incremental Learning without Forgettingcited by 220
  19. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
  20. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  21. 2018Large scale distributed neural network training through online distillationcited by 144
  22. 2020More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classificationcited by 126
  23. 2020Graph Convolutional Networks for Hyperspectral Image Classificationcited by 113
  24. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 99
  25. 2021Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Datacited by 67
  26. 2018An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixingcited by 65
  27. 2020The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classificationcited by 64
  28. 2020MgSvF: Multi-Grained Slow vs. Fast Framework for Few-Shot Class-Incremental Learningcited by 61
  29. 2020ResKD: Residual-Guided Knowledge Distillationcited by 25
  30. 2021When Video Classification Meets Incremental Classescited by 18
  31. 2020FedMAX: Mitigating Activation Divergence for Accurate and Communication-Efficient Federated Learningcited by 14
  32. 2020Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimizationcited by 9
  33. 2020GPCA: A Probabilistic Framework for Gaussian Process Embedded Channel Attentioncited by 7
  34. 2016Ensemble-Compression: A New Method for Parallel Training of Deep Neural Networkscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2.

4 papers.

  1. 2022Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interferencecited by 44
  2. 2023CoDeC: Communication-Efficient Decentralized Continual Learningcited by 2
  3. 2024Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learningcited by 2
  4. 2024Few-Shot Class-Incremental Learning with Non-IID Decentralized Datacited by 0