mlsanityData to 7 Oct 2026

Distributed Inference on Mobile Edge and Cloud: A Data-Cartography based Clustering Approach

arXiv 2412.16616 ↗First posted 21 Dec 2024cs.DC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The large size of DNNs poses a significant challenge for deployment on devices with limited resources, such as mobile, edge, and IoT platforms. To address this issue, a distributed inference framework can be utilized. In this framework, a small-scale DNN (initial layers) is deployed on mobile devices, a larger version on edge devices, and the full DNN on the cloud. Samples with low complexity (easy) can be processed on mobile, those with moderate complexity (medium) on edge devices, and high complexity (hard) samples on the cloud. Given that the complexity of each sample is unknown in advance, the crucial question in distributed inference is determining the sample complexity for appropriate DNN processing. We introduce a novel method named our, which leverages the Data Cartography approach initially proposed for enhancing DNN generalization. By employing data cartography, we assess sample complexity. our aims to boost accuracy while considering the offloading costs from mobile to edge/cloud. Our experimental results on GLUE datasets, covering a variety of NLP tasks, indicate that our approach significantly lowers inference costs by more than 43% while maintaining a minimal accuracy drop of less than 0.5% compared to performing all inferences on the cloud. The source code is available at https://anonymous.4open.science/r/DIMEC-1B04.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 21 Dec 2024.

Builds on

23 of 34 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  6. 2019Are Sixteen Heads Really Better than One?cited by 803
  7. 2020A Survey on Visual Transformercited by 653
  8. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  9. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  10. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  11. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  12. 2021I-BERT: Integer-only BERT Quantizationcited by 250
11 more
  1. 2020TernaryBERT: Distillation-aware Ultra-low Bit BERTcited by 145
  2. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 114
  3. 2019BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systemscited by 94
  4. 2019BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Servicescited by 67
  5. 2020Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networkscited by 28
  6. 2022BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computingcited by 28
  7. 2023SplitEE: Early Exit in Deep Neural Networks with Split Computingcited by 13
  8. 2024CEEBERT: Cross-Domain Inference in Early Exit BERTcited by 12
  9. 2022Unsupervised Early Exit in DNNs with Multiple Exitscited by 10
  10. 2024I-SplitEE: Image classification in Split Computing DNNs with Early Exitscited by 9
  11. 2021Early-exit deep neural networks for distorted images: providing an efficient edge offloadingcited by 8

Built on by

Not cited yet.