# TrimTuner: Efficient Optimization of Machine Learning Jobs in the Cloud via Sub-Sampling

arXiv: https://arxiv.org/abs/2011.04726 · HTML: https://mlsanity.com/paper/2011.04726/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2020.

## Built on by (most cited first, 12 of 12)

- [Multi-Objectivizing Software Configuration Tuning (for a single performance concern)](https://mlsanity.com/paper/2106.01331/) (2021)
- [MMO: Meta Multi-Objectivization for Software Configuration Tuning](https://mlsanity.com/paper/2112.07303/) (2021)
- [Batch Multi-Fidelity Active Learning with Budget Constraints](https://mlsanity.com/paper/2210.12704/) (2022)
- [Lifelong Dynamic Optimization for Self-Adaptive Systems: Fact or Fiction?](https://mlsanity.com/paper/2201.07096/) (2022)
- [Supervising the Multi-Fidelity Race of Hyperparameter Configurations](https://mlsanity.com/paper/2202.09774/) (2022)
- [Adapting Multi-objectivized Software Configuration Tuning](https://mlsanity.com/paper/2404.04744/) (2024)
- [Planning Landscape Analysis for Self-Adaptive Systems](https://mlsanity.com/paper/2203.12472/) (2022)
- [HyperJump: Accelerating HyperBand via Risk Modelling](https://mlsanity.com/paper/2108.02479/) (2021)
- [Perona: Robust Infrastructure Fingerprinting for Resource-Efficient Big Data Analytics](https://mlsanity.com/paper/2211.08227/) (2022)
- [Hyper-parameter Tuning for Adversarially Robust Models](https://mlsanity.com/paper/2304.02497/) (2023)
- [A Trajectory-Based Bayesian Approach to Multi-Objective Hyperparameter Optimization with Epoch-Aware Trade-Offs](https://mlsanity.com/paper/2405.15303/) (2024)
- [Karasu: A Collaborative Approach to Efficient Cluster Configuration for Big Data Analytics](https://mlsanity.com/paper/2308.11792/) (2023)

## Abstract (arXiv)

This work introduces TrimTuner, the first system for optimizing machine learning jobs in the cloud to exploit sub-sampling techniques to reduce the cost of the optimization process while keeping into account user-specified constraints. TrimTuner jointly optimizes the cloud and application-specific parameters and, unlike state of the art works for cloud optimization, eschews the need to train the model with the full training set every time a new configuration is sampled. Indeed, by leveraging sub-sampling techniques and data-sets that are up to 60x smaller than the original one, we show that TrimTuner can reduce the cost of the optimization process by up to 50x. Further, TrimTuner speeds-up the recommendation process by 65x with respect to state of the art techniques for hyper-parameter optimization that use sub-sampling techniques. The reasons for this improvement are twofold: i) a novel domain specific heuristic that reduces the number of configurations for which the acquisition function has to be evaluated; ii) the adoption of an ensemble of decision trees that enables boosting the speed of the recommendation process by one additional order of magnitude.