# p-Meta: Towards On-device Deep Model Adaptation

arXiv: https://arxiv.org/abs/2206.12705 · HTML: https://mlsanity.com/paper/2206.12705/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 25 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML](https://mlsanity.com/paper/1909.09157/) (2019)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [The Reversible Residual Network: Backpropagation Without Storing Activations](https://mlsanity.com/paper/1707.04585/) (2017)
- [Memory-Efficient Backpropagation Through Time](https://mlsanity.com/paper/1606.03401/) (2016)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [Highway and Residual Networks learn Unrolled Iterative Estimation](https://mlsanity.com/paper/1612.07771/) (2016)
- [Torchmeta: A Meta-Learning library for PyTorch](https://mlsanity.com/paper/1909.06576/) (2019)
- [Sparse Weight Activation Training](https://mlsanity.com/paper/2001.01969/) (2020)
- [Learning where to learn: Gradient sparsity in meta and continual learning](https://mlsanity.com/paper/2110.14402/) (2021)
- [Low-rank Gradient Approximation For Memory-Efficient On-device Training of Deep Neural Network](https://mlsanity.com/paper/2001.08885/) (2020)
- [On-device Federated Learning with Flower](https://mlsanity.com/paper/2104.03042/) (2021)

## Built on by (most cited first, 8 of 8)

- [On-device Training: A First Overview on Existing Systems](https://mlsanity.com/paper/2212.00824/) (2022)
- [TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge](https://mlsanity.com/paper/2307.09988/) (2023)
- [LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms](https://mlsanity.com/paper/2311.11420/) (2023)
- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Hierarchical Training of Deep Neural Networks Using Early Exiting](https://mlsanity.com/paper/2303.02384/) (2023)
- [Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Training](https://mlsanity.com/paper/2504.09405/) (2025)
- [AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments](https://mlsanity.com/paper/2410.08256/) (2024)
- [Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers](https://mlsanity.com/paper/2509.15113/) (2025)

## Abstract (arXiv)

Data collected by IoT devices are often private and have a large diversity across users. Therefore, learning requires pre-training a model with available representative data samples, deploying the pre-trained model on IoT devices, and adapting the deployed model on the device with local data. Such an on-device adaption for deep learning empowered applications demands data and memory efficiency. However, existing gradient-based meta learning schemes fail to support memory-efficient adaptation. To this end, we propose p-Meta, a new meta learning method that enforces structure-wise partial parameter updates while ensuring fast generalization to unseen tasks. Evaluations on few-shot image classification and reinforcement learning tasks show that p-Meta not only improves the accuracy but also substantially reduces the peak dynamic memory by a factor of 2.5 on average compared to state-of-the-art few-shot adaptation methods.