mlsanityData to 14 Oct 2026

Towards Experience Replay for Class-Incremental Learning in Fully-Binary Networks

arXiv 2503.07107 ↗First posted 10 Mar 2025cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Binary Neural Networks (BNNs) are a promising approach to enable Artificial Neural Network (ANN) implementation on ultra-low power edge devices. Such devices may compute data in highly dynamic environments, in which the classes targeted for inference can evolve or even novel classes may arise, requiring continual learning. Class Incremental Learning (CIL) is a common type of continual learning for classification problems, that has been scarcely addressed in the context of BNNs. Furthermore, most of existing BNNs models are not fully binary, as they require several real-valued network layers, at the input, the output, and for batch normalization. This paper goes a step further, enabling class incremental learning in Fully-Binarized NNs (FBNNs) through four main contributions. We firstly revisit the FBNN design and its training procedure that is suitable to CIL. Secondly, we explore loss balancing, a method to trade-off the performance of past and current classes. Thirdly, we propose a semi-supervised method to pre-train the feature extractor of the FBNN for transferable representations. Fourthly, two conventional CIL methods, ie, Latent and Native replay, are thoroughly compared. These contributions are exemplified first on the CIFAR100 dataset, before being scaled up to address the CORE50 continual learning benchmark. The final results based on our 3Mb FBNN on CORE50 exhibit at par and better performance than conventional real-valued larger NN models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 49 of its 90 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

49 of 90 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  8. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  11. 2016Progressive Neural Networkscited by 1,361
  12. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
37 more
  1. 2017Continual Learning with Deep Generative Replaycited by 1,158
  2. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  3. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  4. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  5. 2019Large Scale Incremental Learningcited by 732
  6. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  7. 2019Three scenarios for continual learningcited by 614
  8. 2013Network In Networkcited by 605
  9. 2018End-to-End Incremental Learningcited by 597
  10. 2019On Tiny Episodic Memories in Continual Learningcited by 307
  11. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 289
  12. 2023Class-Incremental Learning: A Surveycited by 209
  13. 2020Binary Neural Networks: A Surveycited by 190
  14. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 184
  15. 2020Memory-Efficient Incremental Learning Through Feature Adaptationcited by 107
  16. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  17. 2019Latent Replay for Real-Time Continual Learningcited by 97
  18. 2020Rethinking Experience Replay: a Bag of Tricks for Continual Learningcited by 94
  19. 2021Essentials for Class Incremental Learningcited by 57
  20. 2016Loss-aware Binarization of Deep Networkscited by 56
  21. 2019Rehearsal-Free Continual Learning over Small Non-I.I.D. Batchescited by 46
  22. 2021A comprehensive review of Binary Neural Networkcited by 45
  23. 2018Bi-Real Net: Binarizing Deep Network Towards Real-Network Performancecited by 42
  24. 2021A TinyML Platform for On-Device Continual Learning with Quantized Latent Replayscited by 37
  25. 2018DeeSIL: Deep-Shallow Incremental Learningcited by 30
  26. 2021Synaptic metaplasticity in binarized neural networkscited by 29
  27. 2020Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art Binarized Neural Networkscited by 20
  28. 2021"BNN - BN = ?": Training Binary Neural Networks without Batch Normalizationcited by 17
  29. 2022Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learningcited by 16
  30. 2021S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibrationcited by 15
  31. 2021ROBIN: A Robust Optical Binary Neural Network Acceleratorcited by 15
  32. 2024A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Networkcited by 5
  33. 2024Enabling On-device Continual Learning with Binary Neural Networkscited by 4
  34. 2025Histogram-Equalized Quantization for logic-gated Residual Neural Networkscited by 4
  35. 2023On-Device Learning with Binary Neural Networkscited by 3
  36. 2025A 1Mb mixed-precision quantized encoder for image classification and patch-based compressioncited by 1
  37. 2025BILLNET: A Binarized Conv3D-LSTM Network with Logic-gated residual architecture for hardware-efficient video inferencecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Generative Binary Memory: Pseudo-Replay Class-Incremental Learning on Binarized Embeddings
  2. 2022Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation
  3. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning
  4. 2026Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Experts
  5. 2026Prototype Latent World Model Replay for Class-Incremental Learning
  6. 2022Class-Incremental Continual Learning into the eXtended DER-verse
  7. 2023Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning
  8. 2024Gradient Reweighting: Towards Imbalanced Class-Incremental Learning