# CheapET-3: Cost-Efficient Use of Remote DNN Models

arXiv: https://arxiv.org/abs/2208.11552 · HTML: https://mlsanity.com/paper/2208.11552/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 24 Aug 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Misbehaviour Prediction for Autonomous Driving Systems](https://mlsanity.com/paper/1910.04443/) (2019)
- [Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)](https://mlsanity.com/paper/2205.00664/) (2022)
- [Towards Structured Evaluation of Deep Neural Network Supervisors](https://mlsanity.com/paper/1903.01263/) (2019)
- [Self-Checking Deep Neural Networks in Deployment](https://mlsanity.com/paper/2103.02371/) (2021)
- [DeepGuard: A Framework for Safeguarding Autonomous Driving Systems from Inconsistent Behavior](https://mlsanity.com/paper/2111.09533/) (2021)
- [Benchmarking Safety Monitors for Image Classifiers with Machine Learning](https://mlsanity.com/paper/2110.01232/) (2021)
- [Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring](https://mlsanity.com/paper/2102.00902/) (2021)
- [Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification](https://mlsanity.com/paper/2101.00982/) (2020)
- [A Review and Refinement of Surprise Adequacy](https://mlsanity.com/paper/2103.05939/) (2021)
- [Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networks](https://mlsanity.com/paper/2103.15580/) (2021)
- [Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testing](https://mlsanity.com/paper/2207.10495/) (2022)

## Built on by (most cited first, 1 of 1)

- [Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks](https://mlsanity.com/paper/2304.02654/) (2023)

## Abstract (arXiv)

On complex problems, state of the art prediction accuracy of Deep Neural Networks (DNN) can be achieved using very large-scale models, consisting of billions of parameters. Such models can only be run on dedicated servers, typically provided by a 3rd party service, which leads to a substantial monetary cost for every prediction. We propose a new software architecture for client-side applications, where a small local DNN is used alongside a remote large-scale model, aiming to make easy predictions locally at negligible monetary cost, while still leveraging the benefits of a large model for challenging inputs. In a proof of concept we reduce prediction cost by up to 50% without negatively impacting system accuracy.