# Amazon SageMaker Automatic Model Tuning: Scalable Gradient-Free Optimization

arXiv: https://arxiv.org/abs/2012.08489 · HTML: https://mlsanity.com/paper/2012.08489/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2020.

## Built on by (most cited first, 9 of 9)

- [Automated Reinforcement Learning (AutoRL): A Survey and Open Problems](https://mlsanity.com/paper/2201.03916/) (2022)
- [Distributionally Robust Bayesian Optimization with varphi-divergences](https://mlsanity.com/paper/2203.02128/) (2022)
- [IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective](https://mlsanity.com/paper/2209.08018/) (2022)
- [Transfer Learning based Search Space Design for Hyperparameter Tuning](https://mlsanity.com/paper/2206.02511/) (2022)
- [Transfer Learning for Fine-grained Classification Using Semi-supervised Learning and Visual Transformers](https://mlsanity.com/paper/2305.10018/) (2023)
- [SigOpt Mulch: An Intelligent System for AutoML of Gradient Boosted Trees](https://mlsanity.com/paper/2307.04849/) (2023)
- [A Simple and Fast Baseline for Tuning Large XGBoost Models](https://mlsanity.com/paper/2111.06924/) (2021)
- [Comparative Analysis of AWS Model Deployment Services](https://mlsanity.com/paper/2405.08175/) (2024)
- [FLStore: Efficient Federated Learning Storage for non-training workloads](https://mlsanity.com/paper/2503.00323/) (2025)

## Abstract (arXiv)

Tuning complex machine learning systems is challenging. Machine learning typically requires to set hyperparameters, be it regularization, architecture, or optimization parameters, whose tuning is critical to achieve good predictive performance. To democratize access to machine learning systems, it is essential to automate the tuning. This paper presents Amazon SageMaker Automatic Model Tuning (AMT), a fully managed system for gradient-free optimization at scale. AMT finds the best version of a trained machine learning model by repeatedly evaluating it with different hyperparameter configurations. It leverages either random search or Bayesian optimization to choose the hyperparameter values resulting in the best model, as measured by the metric chosen by the user. AMT can be used with built-in algorithms, custom algorithms, and Amazon SageMaker pre-built containers for machine learning frameworks. We discuss the core functionality, system architecture, our design principles, and lessons learned. We also describe more advanced features of AMT, such as automated early stopping and warm-starting, showing in experiments their benefits to users.