# Automatic model training under restrictive time constraints

arXiv: https://arxiv.org/abs/2104.10746 · HTML: https://mlsanity.com/paper/2104.10746/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 63 references that are in the map.
- First posted on arXiv 21 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Freeze-Thaw Bayesian Optimization](https://mlsanity.com/paper/1406.3896/) (2014)
- [OBOE: Collaborative Filtering for AutoML Model Selection](https://mlsanity.com/paper/1808.03233/) (2018)
- [Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis](https://mlsanity.com/paper/1812.04300/) (2018)
- [Efficient Rollout Strategies for Bayesian Optimization](https://mlsanity.com/paper/2002.10539/) (2020)
- [Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications](https://mlsanity.com/paper/1812.05916/) (2018)
- [Probabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter Settings](https://mlsanity.com/paper/1910.04522/) (2019)
- [Regress-Later Monte Carlo for optimal control of Markov processes](https://mlsanity.com/paper/1712.09705/) (2017)

## Built on by (most cited first, 1 of 1)

- [Closed-Form Beta Distribution Estimation from Sparse Statistics with Random Forest Implicit Regularization](https://mlsanity.com/paper/2507.23767/) (2025)

## Abstract (arXiv)

We develop a hyperparameter optimisation algorithm, Automated Budget Constrained Training (AutoBCT), which balances the quality of a model with the computational cost required to tune it. The relationship between hyperparameters, model quality and computational cost must be learnt and this learning is incorporated directly into the optimisation problem. At each training epoch, the algorithm decides whether to terminate or continue training, and, in the latter case, what values of hyperparameters to use. This decision weighs optimally potential improvements in the quality with the additional training time and the uncertainty about the learnt quantities. The performance of our algorithm is verified on a number of machine learning problems encompassing random forests and neural networks. Our approach is rooted in the theory of Markov decision processes with partial information and we develop a numerical method to compute the value function and an optimal strategy.