# TASTE: Throughput-Aware Batch Size Tuning for On-Device Edge Learning

arXiv: https://arxiv.org/abs/2609.07444 · HTML: https://mlsanity.com/paper/2609.07444/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 7 Sep 2026.

## Builds on (most cited first, 17 of 17)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Understanding the Role of Training Regimes in Continual Learning](https://mlsanity.com/paper/2006.06958/) (2020)
- [Class-Incremental Continual Learning into the eXtended DER-verse](https://mlsanity.com/paper/2201.00766/) (2022)
- [A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays](https://mlsanity.com/paper/2110.10486/) (2021)
- [DyNet: The Dynamic Neural Network Toolkit](https://mlsanity.com/paper/1701.03980/) (2017)
- [Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensors](https://mlsanity.com/paper/2408.12481/) (2024)
- [On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors](https://mlsanity.com/paper/2403.04071/) (2024)
- [On-Device Training Empowered Transfer Learning For Human Activity Recognition](https://mlsanity.com/paper/2407.03644/) (2024)
- [ED-Batch: Efficient Automatic Batching of Dynamic Neural Networks via Learned Finite State Machines](https://mlsanity.com/paper/2302.03851/) (2023)
- [ACRoBat: Optimizing Auto-batching of Dynamic Deep Learning at Compile Time](https://mlsanity.com/paper/2305.10611/) (2023)
- [Tensor Program Optimization for the RISC-V Vector Extension Using Probabilistic Programs](https://mlsanity.com/paper/2507.01457/) (2025)
- [On-Device Adaptive Battery Power Prediction for Electric Vehicles](https://mlsanity.com/paper/2607.09400/) (2026)

## Abstract (arXiv)

The rise of privacy-preserving artificial intelligence (AI) has shifted the focus of model adaptation and personalization towards on-device learning, where deep learning models are finetuned directly on edge hardware using local user data. However, this shift requires optimization of deep learning training on resource-constrained hardware to maximize throughput while maintaining predictive accuracy. This paper introduces a novel technique for on-device model training that incorporates an efficient Bayesian optimization-based batch size tuning approach to maximize hardware throughput. To evaluate the impact of this hyperparameter on the learning dynamics, we investigated two distinct paradigms: standard supervised learning (SL) and online continual learning (CL). Experimental results across various edge devices demonstrate a throughput ceiling, beyond which increasing the batch size yields no additional throughput gains. The proposed tuning approach identifies the optimal batch size, which, when combined with gradient accumulation and linear learning rate scaling, achieves up to a 2X increase in training throughput on platforms such as Raspberry Pi 4 compared to maximum batch sizes, without compromising model accuracy. Furthermore, in the CL paradigm, we demonstrate that optimal batch sizes maintain the stability-plasticity balance required for incremental learning, effectively mitigating catastrophic forgetting while maximizing computational efficiency on edge-hardware.