# Efficient On-device Training via Gradient Filtering

arXiv: https://arxiv.org/abs/2301.00330 · HTML: https://mlsanity.com/paper/2301.00330/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 1 Jan 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [Surgical Fine-Tuning Improves Adaptation to Distribution Shifts](https://mlsanity.com/paper/2210.11466/) (2022)
- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [Scalable Methods for 8-bit Training of Neural Networks](https://mlsanity.com/paper/1805.11046/) (2018)
- [FastDepth: Fast Monocular Depth Estimation on Embedded Systems](https://mlsanity.com/paper/1903.03273/) (2019)
- [A Statistical Framework for Low-bitwidth Training of Deep Neural Networks](https://mlsanity.com/paper/2010.14298/) (2020)
- [E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savings](https://mlsanity.com/paper/1910.13349/) (2019)
- [TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning](https://mlsanity.com/paper/2007.11622/) (2020)
- [Distribution Adaptive INT8 Quantization for Training CNNs](https://mlsanity.com/paper/2102.04782/) (2021)

## Built on by (most cited first, 13 of 13)

- [Multisize Dataset Condensation](https://mlsanity.com/paper/2403.06075/) (2024)
- [Efficient Low-rank Backpropagation for Vision Transformer Adaptation](https://mlsanity.com/paper/2309.15275/) (2023)
- [Activation Map Compression through Tensor Decomposition for Deep Learning](https://mlsanity.com/paper/2411.06346/) (2024)
- [Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning](https://mlsanity.com/paper/2505.05086/) (2025)
- [SCoTTi: Save Computation at Training Time with an adaptive framework](https://mlsanity.com/paper/2312.12483/) (2023)
- [Advancing On-Device Neural Network Training with TinyPropv2: Dynamic, Sparse, and Efficient Backpropagation](https://mlsanity.com/paper/2409.07109/) (2024)
- [A General and Efficient Training for Transformer via Token Expansion](https://mlsanity.com/paper/2404.00672/) (2024)
- [Center-Sensitive Kernel Optimization for Efficient On-Device Incremental Learning](https://mlsanity.com/paper/2406.08830/) (2024)
- [Accelerated Training on Low-Power Edge Devices](https://mlsanity.com/paper/2502.18323/) (2025)
- [Study of Training Dynamics for Memory-Constrained Fine-Tuning](https://mlsanity.com/paper/2510.19675/) (2025)
- [EON-1: A Brain-Inspired Processor for Near-Sensor Extreme Edge Online Feature Extraction](https://mlsanity.com/paper/2406.17285/) (2024)
- [Efficient Resource-Constrained Training of Transformers via Subspace Optimization](https://mlsanity.com/paper/2510.09160/) (2025)
- [On-Device Neural Architecture Search](https://mlsanity.com/paper/2606.24900/) (2026)

## Abstract (arXiv)

Despite its importance for federated learning, continuous learning and many other applications, on-device training remains an open problem for EdgeAI. The problem stems from the large number of operations (e.g., floating point multiplications and additions) and memory consumption required during training by the back-propagation algorithm. Consequently, in this paper, we propose a new gradient filtering approach which enables on-device CNN model training. More precisely, our approach creates a special structure with fewer unique elements in the gradient map, thus significantly reducing the computational complexity and memory consumption of back propagation during training. Extensive experiments on image classification and semantic segmentation with multiple CNN models (e.g., MobileNet, DeepLabV3, UPerNet) and devices (e.g., Raspberry Pi and Jetson Nano) demonstrate the effectiveness and wide applicability of our approach. For example, compared to SOTA, we achieve up to 19× speedup and 77.1% memory savings on ImageNet classification with only 0.1% accuracy loss. Finally, our method is easy to implement and deploy; over 20× speedup and 90% energy savings have been observed compared to highly optimized baselines in MKLDNN and CUDNN on NVIDIA Jetson Nano. Consequently, our approach opens up a new direction of research with a huge potential for on-device training.