mlsanityData to 7 Oct 2026

Efficient IoT Inference via Context-Awareness

arXiv 2310.19112 ↗First posted 29 Oct 2023cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

While existing strategies to execute deep learning-based classification on low-power platforms assume the models are trained on all classes of interest, this paper posits that adopting context-awareness i.e. narrowing down a classification task to the current deployment context consisting of only recent inference queries can substantially enhance performance in resource-constrained environments. We propose a new paradigm, CACTUS, for scalable and efficient context-aware classification where a micro-classifier recognizes a small set of classes relevant to the current context and, when context change happens (e.g., a new class comes into the scene), rapidly switches to another suitable micro-classifier. CACTUS features several innovations, including optimizing the training cost of context-aware classifiers, enabling on-the-fly context-aware switching between classifiers, and balancing context switching costs and performance gains via simple yet effective switching policies. We show that CACTUS achieves significant benefits in accuracy, latency, and compute budget across a range of datasets and IoT platforms.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 57 references that are in the map. First posted on arXiv 29 Oct 2023.

Builds on

29 of 57 references.

References · 29

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  11. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  12. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
17 more
  1. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  2. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  3. 2020Better Aggregation in Test-Time Augmentationcited by 121
  4. 2017Distributed Deep Neural Networks over the Cloud, the Edge and End Devicescited by 119
  5. 2018CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUscited by 116
  6. 2018Direct Uncertainty Prediction for Medical Second Opinionscited by 61
  7. 2018Deep feature compression for collaborative object detectioncited by 48
  8. 2018Focus: Querying Large Video Datasets with Low Latency and Low Costcited by 46
  9. 2020Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examplescited by 40
  10. 2019Density estimation in representation space to predict model uncertaintycited by 27
  11. 2018Inhibited Softmax for Uncertainty Estimation in Neural Networkscited by 19
  12. 2021It's always personal: Using Early Exits for Efficient On-Device CNN Personalisationcited by 18
  13. 2016Fast Video Classification via Adaptive Cascading of Deep Modelscited by 17
  14. 2021Lightweight compression of neural network feature tensors for collaborative intelligencecited by 17
  15. 2018FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devicescited by 14
  16. 2020Robustifying the Deployment of tinyML Models for Autonomous mini-vehiclescited by 10
  17. 2021Packet-Loss-Tolerant Split Inference for Delay-Sensitive Deep Learning in Lossy Wireless Networkscited by 7

Built on by

Not cited yet.