# Learning to Rank Learning Curves

arXiv: https://arxiv.org/abs/2006.03361 · HTML: https://mlsanity.com/paper/2006.03361/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2020.

## Built on by (most cited first, 16 of 16)

- [Scaling Laws for Hyperparameter Optimization](https://mlsanity.com/paper/2302.00441/) (2023)
- [Supervising the Multi-Fidelity Race of Hyperparameter Configurations](https://mlsanity.com/paper/2202.09774/) (2022)
- [T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imaging](https://mlsanity.com/paper/2111.07535/) (2021)
- [Optimizing Hyperparameters with Conformal Quantile Regression](https://mlsanity.com/paper/2305.03623/) (2023)
- [Learning to Limit Data Collection via Scaling Laws: A Computational Interpretation for the Legal Principle of Data Minimization](https://mlsanity.com/paper/2107.08096/) (2021)
- [Deep Ranking Ensembles for Hyperparameter Optimization](https://mlsanity.com/paper/2303.15212/) (2023)
- [Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics](https://mlsanity.com/paper/2201.04194/) (2022)
- [Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architectures](https://mlsanity.com/paper/2207.05321/) (2022)
- [HyT-NAS: Hybrid Transformers Neural Architecture Search for Edge Devices](https://mlsanity.com/paper/2303.04440/) (2023)
- [Cost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolation](https://mlsanity.com/paper/2405.17918/) (2024)
- [Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning](https://mlsanity.com/paper/2510.21379/) (2025)
- [Configuration-to-Performance Scaling Law with Neural Ansatz](https://mlsanity.com/paper/2602.10300/) (2026)
- [Learning to Rank Normalized Entropy Curves with Differentiable Window Transformation](https://mlsanity.com/paper/2301.10443/) (2023)
- [Meta-Learning from Learning Curves for Budget-Limited Algorithm Selection](https://mlsanity.com/paper/2410.07696/) (2024)
- [OptiProxy-NAS: Optimization Proxy based End-to-End Neural Architecture Search](https://mlsanity.com/paper/2509.05656/) (2025)
- [Efficient Hyperparameter Search for Non-Stationary Model Training](https://mlsanity.com/paper/2512.01258/) (2025)

## Abstract (arXiv)

Many automated machine learning methods, such as those for hyperparameter and neural architecture optimization, are computationally expensive because they involve training many different model configurations. In this work, we present a new method that saves computational budget by terminating poor configurations early on in the training. In contrast to existing methods, we consider this task as a ranking and transfer learning problem. We qualitatively show that by optimizing a pairwise ranking loss and leveraging learning curves from other datasets, our model is able to effectively rank learning curves without having to observe many or very long learning curves. We further demonstrate that our method can be used to accelerate a neural architecture search by a factor of up to 100 without a significant performance degradation of the discovered architecture. In further experiments we analyze the quality of ranking, the influence of different model components as well as the predictive behavior of the model.