# Tune: A Research Platform for Distributed Model Selection and Training

arXiv: https://arxiv.org/abs/1807.05118 · HTML: https://mlsanity.com/paper/1807.05118/

## Key facts

- Cited by 395 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2018.

## Built on by (most cited first, 30 of 395)

- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [GRAND: Graph Neural Diffusion](https://mlsanity.com/paper/2106.10934/) (2021)
- [Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better](https://mlsanity.com/paper/2106.08962/) (2021)
- [Scene Text Recognition with Permuted Autoregressive Sequence Models](https://mlsanity.com/paper/2207.06966/) (2022)
- [Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance](https://mlsanity.com/paper/2304.11127/) (2023)
- [MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages](https://mlsanity.com/paper/2204.08582/) (2022)
- [Beltrami Flow and Neural Diffusion on Graphs](https://mlsanity.com/paper/2110.09443/) (2021)
- [Towards Learning Universal Hyperparameter Optimizers with Transformers](https://mlsanity.com/paper/2205.13320/) (2022)
- [Scalable Membership Inference Attacks via Quantile Regression](https://mlsanity.com/paper/2307.03694/) (2023)
- [Hyperparameters in Reinforcement Learning and How To Tune Them](https://mlsanity.com/paper/2306.01324/) (2023)
- [Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity](https://mlsanity.com/paper/2305.08252/) (2023)
- [Model Zoo: A Growing "Brain" That Learns Continually](https://mlsanity.com/paper/2106.03027/) (2021)
- [Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding](https://mlsanity.com/paper/2103.04850/) (2021)
- [Towards Scalable and Versatile Weight Space Learning](https://mlsanity.com/paper/2406.09997/) (2024)
- [Aligned Diffusion Schr"odinger Bridges](https://mlsanity.com/paper/2302.11419/) (2023)
- [SizeShiftReg: a Regularization Method for Improving Size-Generalization in Graph Neural Networks](https://mlsanity.com/paper/2207.07888/) (2022)
- [ThinkSwitcher: When to Think Hard, When to Think Fast](https://mlsanity.com/paper/2505.14183/) (2025)
- [ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets](https://mlsanity.com/paper/2110.14771/) (2021)
- [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://mlsanity.com/paper/2402.14989/) (2024)
- [OmniPred: Language Models as Universal Regressors](https://mlsanity.com/paper/2402.14547/) (2024)
- [CUDA: Curriculum of Data Augmentation for Long-Tailed Recognition](https://mlsanity.com/paper/2302.05499/) (2023)
- [Credit Assignment in Neural Networks through Deep Feedback Control](https://mlsanity.com/paper/2106.07887/) (2021)
- [Learning 3D Representations of Molecular Chirality with Invariance to Bond Rotations](https://mlsanity.com/paper/2110.04383/) (2021)
- [DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Training](https://mlsanity.com/paper/2305.12127/) (2023)
- [AutoVP: An Automated Visual Prompting Framework and Benchmark](https://mlsanity.com/paper/2310.08381/) (2023)
- [Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data](https://mlsanity.com/paper/2111.02275/) (2021)
- [Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluation](https://mlsanity.com/paper/2307.01680/) (2023)
- [Open Materials Generation with Stochastic Interpolants](https://mlsanity.com/paper/2502.02582/) (2025)
- [Checkpoint Merging via Bayesian Optimization in LLM Pretraining](https://mlsanity.com/paper/2403.19390/) (2024)

## Abstract (arXiv)

Modern machine learning algorithms are increasingly computationally demanding, requiring specialized hardware and distributed computation to achieve high performance in a reasonable time frame. Many hyperparameter search algorithms have been proposed for improving the efficiency of model selection, however their adaptation to the distributed compute environment is often ad-hoc. We propose Tune, a unified framework for model selection and training that provides a narrow-waist interface between training scripts and search algorithms. We show that this interface meets the requirements for a broad range of hyperparameter search algorithms, allows straightforward scaling of search to large clusters, and simplifies algorithm implementation. We demonstrate the implementation of several state-of-the-art hyperparameter search algorithms in Tune. Tune is available at http://ray.readthedocs.io/en/latest/tune.html.