mlsanityData to 7 Oct 2026

Monitoring and Adapting ML Models on Mobile Devices

arXiv 2305.07772 ↗First posted 12 May 2023cs.LG cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

ML models are increasingly being pushed to mobile devices, for low-latency inference and offline operation. However, once the models are deployed, it is hard for ML operators to track their accuracy, which can degrade unpredictably (e.g., due to data drift). We design the first end-to-end system for continuously monitoring and adapting models on mobile devices without requiring feedback from users. Our key observation is that often model degradation is due to a specific root cause, which may affect a large group of devices. Therefore, once the system detects a consistent degradation across a large number of devices, it employs a root cause analysis to determine the origin of the problem and applies a cause-specific adaptation. We evaluate the system on two computer vision datasets, and show it consistently boosts accuracy compared to existing approaches. On a dataset containing photos collected from driving cars, our system improves the accuracy on average by 15%.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 40 of its 88 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

40 of 88 references.

References · 40

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2016Learning without Forgettingcited by 2,465
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  9. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  10. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  11. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  12. 2016End to End Learning for Self-Driving Carscited by 1,000
28 more
  1. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  2. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  3. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 511
  4. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  5. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  6. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  7. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  8. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  9. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  10. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  11. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 240
  12. 2017Incremental Learning of Object Detectors without Catastrophic Forgettingcited by 208
  13. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  14. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  15. 2018Continuous Learning in Single-Incremental-Task Scenarioscited by 109
  16. 2019Single Image Deraining: A Comprehensive Benchmark Analysiscited by 84
  17. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  18. 2020Tent: Fully Test-time Adaptation by Entropy Minimizationcited by 59
  19. 2020Ekya: Continuous Learning of Video Analytics Models on Edge Compute Serverscited by 58
  20. 2019Federated Learning Of Out-Of-Vocabulary Wordscited by 51
  21. 2022Operationalizing Machine Learning: An Interview Studycited by 25
  22. 2023FLINT: A Platform for Federated Learning Integrationcited by 17
  23. 2021Towards Observability for Production Machine Learning Pipelinescited by 10
  24. 2020Characterizing and Taming Model Instability Across Edge Devicescited by 7
  25. 2021ML-EXray: Visibility into ML Deployment on the Edgecited by 6
  26. 2022A Tale of Two Models: Constructing Evasive Attacks on Edge Modelscited by 6
  27. 2022Rethinking Streaming Machine Learning Evaluationcited by 5
  28. 2023Moving Fast With Broken Datacited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 1.

6 papers.

  1. 2023Equilibrium in the Computing Continuum through Active Inferencecited by 9
  2. 2024Distributed Monitoring for Data Distribution Shifts in Edge-ML Fraud Detectioncited by 1
  3. 2024DEBUG-HD: Debugging TinyML models on-device using Hyper-Dimensional computingcited by 1
  4. 2025Chorus: Harmonizing Context and Sensing Signals for Data-Free Model Customization in IoTcited by 1
  5. 2025NuWa: Deriving Lightweight Class-Specific Vision Transformers for Edge Devicescited by 0
  6. 2026DriftGuard: Mitigating Asynchronous Data Drift in Federated Learningcited by 0