# FrugalML: How to Use ML Prediction APIs More Accurately and Cheaply

arXiv: https://arxiv.org/abs/2006.07512 · HTML: https://mlsanity.com/paper/2006.07512/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 30 of 33)

- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [Large Language Model Routing with Benchmark Datasets](https://mlsanity.com/paper/2309.15789/) (2023)
- [Towards Unbiased and Accurate Deferral to Multiple Experts](https://mlsanity.com/paper/2102.13004/) (2021)
- [Fusing Models with Complementary Expertise](https://mlsanity.com/paper/2310.01542/) (2023)
- [MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs](https://mlsanity.com/paper/2407.10834/) (2024)
- [What is the Role of Small Models in the LLM Era: A Survey](https://mlsanity.com/paper/2409.06857/) (2024)
- [CARROT: A Cost Aware Rate Optimal Router](https://mlsanity.com/paper/2502.03261/) (2025)
- [Optimizing Model Selection for Compound AI Systems](https://mlsanity.com/paper/2502.14815/) (2025)
- [Towards Optimizing the Costs of LLM Usage](https://mlsanity.com/paper/2402.01742/) (2024)
- [Data Pricing in Machine Learning Pipelines](https://mlsanity.com/paper/2108.07915/) (2021)
- [Agreement-Based Cascading for Efficient Inference](https://mlsanity.com/paper/2407.02348/) (2024)
- [MLDemon: Deployment Monitoring for Machine Learning Systems](https://mlsanity.com/paper/2104.13621/) (2021)
- [Creating Training Sets via Weak Indirect Supervision](https://mlsanity.com/paper/2110.03484/) (2021)
- [Intelligence per Watt: Measuring Intelligence Efficiency of Local AI](https://mlsanity.com/paper/2511.07885/) (2025)
- [Did the Model Change? Efficiently Assessing Machine Learning API Shifts](https://mlsanity.com/paper/2107.14203/) (2021)
- [Efficient Online ML API Selection for Multi-Label Classification Tasks](https://mlsanity.com/paper/2102.09127/) (2021)
- [HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions](https://mlsanity.com/paper/2209.08443/) (2022)
- [Active Assessment of Prediction Services as Accuracy Surface Over Attribute Combinations](https://mlsanity.com/paper/2108.06514/) (2021)
- [Mondrian: Prompt Abstraction Attack Against Large Language Models for Cheaper API Pricing](https://mlsanity.com/paper/2308.03558/) (2023)
- [CascadeServe: Unlocking Model Cascades for Inference Serving](https://mlsanity.com/paper/2406.14424/) (2024)
- [OCCAM: Towards Cost-Efficient and Accuracy-Aware Classification Inference](https://mlsanity.com/paper/2406.04508/) (2024)
- [Optimizing Large Language Models: Metrics, Energy Efficiency, and Case Study Insights](https://mlsanity.com/paper/2504.06307/) (2025)
- [RouteNLP: Closed-Loop LLM Routing with Conformal Cascading and Distillation Co-Optimization](https://mlsanity.com/paper/2604.23577/) (2026)
- [Cocktail: Leveraging Ensemble Learning for Optimized Model Serving in Public Cloud](https://mlsanity.com/paper/2106.05345/) (2021)
- [Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models](https://mlsanity.com/paper/2411.09837/) (2024)
- [Uncertainty-Aware Sequential Decision Rules for Event-Triggered LLM Invocation in Streaming Systems](https://mlsanity.com/paper/2607.13048/) (2026)
- [Automatic and Efficient Customization of Neural Networks for ML Applications](https://mlsanity.com/paper/2310.04685/) (2023)
- [How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness](https://mlsanity.com/paper/2607.25068/) (2026)
- [Multinomial Subset Routing with Sum-Max Rewards and Operational Constraints](https://mlsanity.com/paper/2608.16375/) (2026)

## Abstract (arXiv)

Prediction APIs offered for a fee are a fast-growing industry and an important part of machine learning as a service. While many such services are available, the heterogeneity in their price and performance makes it challenging for users to decide which API or combination of APIs to use for their own data and budget. We take a first step towards addressing this challenge by proposing FrugalML, a principled framework that jointly learns the strength and weakness of each API on different data, and performs an efficient optimization to automatically identify the best sequential strategy to adaptively use the available APIs within a budget constraint. Our theoretical analysis shows that natural sparsity in the formulation can be leveraged to make FrugalML efficient. We conduct systematic experiments using ML APIs from Google, Microsoft, Amazon, IBM, Baidu and other providers for tasks including facial emotion recognition, sentiment analysis and speech recognition. Across various tasks, FrugalML can achieve up to 90% cost reduction while matching the accuracy of the best single API, or up to 5% better accuracy while matching the best API's cost.