mlsanityData to 14 Oct 2026

Mandheling: Mixed-Precision On-Device DNN Training with DSP Offloading

arXiv 2206.07509 ↗First posted 15 Jun 2022cs.NIcited by 29

Not on the map. Why?

Abstract (arXiv)

This paper proposes Mandheling, the first system that enables highly resource-efficient on-device training by orchestrating the mixed-precision training with on-chip Digital Signal Processing (DSP) offloading. Mandheling fully explores the advantages of DSP in integer-based numerical calculation by four novel techniques: (1) a CPU-DSP co-scheduling scheme to mitigate the overhead from DSP-unfriendly operators; (2) a self-adaptive rescaling algorithm to reduce the overhead of dynamic rescaling in backward propagation; (3) a batch-splitting algorithm to improve the DSP cache efficiency; (4) a DSP-compute subgraph reusing mechanism to eliminate the preparation overhead on DSP. We have fully implemented Mandheling and demonstrate its effectiveness through extensive experiments. The results show that, compared to the state-of-the-art DNN engines from TFLite and MNN, Mandheling reduces the per-batch training time by 5.5× and the energy consumption by 8.9× on average. In end-to-end training tasks, Mandheling reduces up to 10.7× convergence time and 13.1× energy consumption, with only 1.9%-2.7% accuracy loss compared to the FP32 precision setting.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Jun 2022.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 10; 2025: 10; 2026: 3.

29 papers.

Sort:

By area:

  1. 2024Fast On-device LLM Inference with NPUscited by 62
  2. 2024Tiny Machine Learning: Progress and Futurescited by 40
  3. 2022On-device Training: A First Overview on Existing Systemscited by 19
  4. 2023TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edgecited by 18
  5. 2022Federated Few-Shot Learning for Mobile NLPcited by 15
  6. 2024Asteroid: Resource-Efficient Hybrid Pipeline Parallelism for Collaborative DNN Training on Heterogeneous Edge Devicescited by 13
  7. 2023ElasticTrainer: Speeding Up On-Device Training with Runtime Elastic Tensor Selectioncited by 12
  8. 2024Lifelong Intelligence Beyond the Edge using Hyperdimensional Computingcited by 5
  9. 2024On-Device Training of Fully Quantized Deep Neural Networks on Cortex-M Microcontrollerscited by 5
  10. 2025From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devicescited by 4
  11. 2024Low-Energy On-Device Personalization for MCUscited by 3
  12. 2024Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligencecited by 3
17 more
  1. 2025FedEL: Federated Elastic Learning for Heterogeneous Devicescited by 3
  2. 2025Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Trainingcited by 2
  3. 2025Recursive Offloading for LLM Serving in Multi-tier Networkscited by 2
  4. 2025A Two-Stage Data Selection Framework for Data-Efficient Model Training on Edge Devicescited by 2
  5. 2023Breaking On-device Training Memory Wall: A Systematic Surveycited by 1
  6. 2023Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environmentscited by 1
  7. 2024Resource-Efficient Personal Large Language Models Fine-Tuning with Collaborative Edge Computingcited by 1
  8. 2024AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environmentscited by 1
  9. 2025LeanTTA: A Backpropagation-Free and Stateless Approach to Quantized Test-Time Adaptation on Edge Devicescited by 1
  10. 2025Optimizing Storage Overhead of User Behavior Log for ML-embedded Mobile Appscited by 1
  11. 2024Online Resource Allocation for Edge Intelligence with Colocated Model Retraining and Inferencecited by 0
  12. 2025THOR: A Generic Energy Estimation Approach for On-Device Trainingcited by 0
  13. 2025Moss: Proxy Model-based Full-Weight Aggregation in Federated Learning with Heterogeneous Modelscited by 0
  14. 2025Comprehensive Design Space Exploration for Tensorized Neural Network Hardware Acceleratorscited by 0
  15. 2026Optimizing Feature Extraction for On-device Model Inference with User Behavior Sequencescited by 0
  16. 2026Beyond Benchmarks: Continuous Edge Inference for Fine-Grained Roadside Perceptioncited by 0
  17. 2026FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUscited by 0