# Center-Sensitive Kernel Optimization for Efficient On-Device Incremental Learning

arXiv: https://arxiv.org/abs/2406.08830 · HTML: https://mlsanity.com/paper/2406.08830/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [Rainbow Memory: Continual Learning with a Memory of Diverse Samples](https://mlsanity.com/paper/2103.17230/) (2021)
- [Compacting, Picking and Growing for Unforgetting Continual Learning](https://mlsanity.com/paper/1910.06562/) (2019)
- [Simple but Effective: CLIP Embeddings for Embodied AI](https://mlsanity.com/paper/2111.09888/) (2021)
- [Understanding the Role of Training Regimes in Continual Learning](https://mlsanity.com/paper/2006.06958/) (2020)
- [ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks](https://mlsanity.com/paper/1908.03930/) (2019)
- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [Training Networks in Null Space of Feature Covariance for Continual Learning](https://mlsanity.com/paper/2103.07113/) (2021)
- [Scalable Methods for 8-bit Training of Neural Networks](https://mlsanity.com/paper/1805.11046/) (2018)
- [Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks](https://mlsanity.com/paper/2303.03667/) (2023)
- [Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning](https://mlsanity.com/paper/2203.06359/) (2022)
- [GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning](https://mlsanity.com/paper/2111.11210/) (2021)
- [MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge](https://mlsanity.com/paper/2110.14032/) (2021)
- [Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.08566/) (2023)
- [SparCL: Sparse Continual Learning on the Edge](https://mlsanity.com/paper/2209.09476/) (2022)
- [A Statistical Framework for Low-bitwidth Training of Deep Neural Networks](https://mlsanity.com/paper/2010.14298/) (2020)
- [When to Prune? A Policy towards Early Structural Pruning](https://mlsanity.com/paper/2110.12007/) (2021)
- [Wanderlust: Online Continual Object Detection in the Real World](https://mlsanity.com/paper/2108.11005/) (2021)
- [Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector](https://mlsanity.com/paper/2110.07905/) (2021)
- [Interactron: Embodied Adaptive Object Detection](https://mlsanity.com/paper/2202.00660/) (2022)
- [Efficient On-device Training via Gradient Filtering](https://mlsanity.com/paper/2301.00330/) (2023)
- [D3Former: Debiased Dual Distilled Transformer for Incremental Learning](https://mlsanity.com/paper/2208.00777/) (2022)
- [Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Models](https://mlsanity.com/paper/2307.15049/) (2023)

## Built on by (most cited first, 2 of 2)

- [StPR: Spatiotemporal Preservation and Routing for Exemplar-Free Video Class-Incremental Learning](https://mlsanity.com/paper/2505.13997/) (2025)
- [EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2506.12351/) (2025)

## Abstract (arXiv)

To facilitate the evolution of edge intelligence in ever-changing environments, we study on-device incremental learning constrained in limited computation resource in this paper. Current on-device training methods just focus on efficient training without considering the catastrophic forgetting, preventing the model getting stronger when continually exploring the world. To solve this problem, a direct solution is to involve the existing incremental learning mechanisms into the on-device training framework. Unfortunately, such a manner cannot work well as those mechanisms usually introduce large additional computational cost to the network optimization process, which would inevitably exceed the memory capacity of the edge devices. To address this issue, this paper makes an early effort to propose a simple but effective edge-friendly incremental learning framework. Based on an empirical study on the knowledge intensity of the kernel elements of the neural network, we find that the center kernel is the key for maximizing the knowledge intensity for learning new data, while freezing the other kernel elements would get a good balance on the model's capacity for overcoming catastrophic forgetting. Upon this finding, we further design a center-sensitive kernel optimization framework to largely alleviate the cost of the gradient computation and back-propagation. Besides, a dynamic channel element selection strategy is also proposed to facilitate a sparse orthogonal gradient projection for further reducing the optimization complexity, upon the knowledge explored from the new task data. Extensive experiments validate our method is efficient and effective, e.g., our method achieves average accuracy boost of 38.08% with even less memory and approximate computation compared to existing on-device training methods, indicating its significant potential for on-device incremental learning.