mlsanityData to 7 Oct 2026

Partial Rankings of Optimizers

arXiv 2402.16565 ↗First posted 26 Feb 2024cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce a framework for benchmarking optimizers according to multiple criteria over various test functions. Based on a recently introduced union-free generic depth function for partial orders/rankings, it fully exploits the ordinal information and allows for incomparability. Our method describes the distribution of all partial orders/rankings, avoiding the notorious shortcomings of aggregation. This permits to identify test functions that produce central or outlying rankings of optimizers and to assess the quality of benchmarking suites.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 11 of its 39 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

11 of 39 references.

References · 11

  1. 2016COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Settingcited by 135
  2. 2019MLPerf Training Benchmarkcited by 114
  3. 2022PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithmcited by 51
  4. 2023Benchmarking Neural Network Training Algorithmscited by 46
  5. 2019DeepOBS: A Deep Learning Optimizer Benchmark Suitecited by 37
  6. 2023Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RLcited by 36
  7. 2022Statistical Comparisons of Classifiers by Generalized Stochastic Dominancecited by 20
  8. 2020Choosing among notions of multivariate depth statisticscited by 14
  9. 2023Robust Statistical Comparison of Random Variables with Locally Varying Scale of Measurementcited by 13
  10. 2022Min-Max Bilevel Multi-objective Optimization with Applications in Machine Learningcited by 9
  11. 2023Depth Functions for Partial Orders with a Descriptive Analysis of Machine Learning Algorithmscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 3.

8 papers.

  1. 2024Explaining Bayesian Optimization by Shapley Values Facilitates Human-AI Collaborationcited by 12
  2. 2024Statistical Multicriteria Benchmarking via the GSD-Frontcited by 11
  3. 2025Statistical Multicriteria Evaluation of LLM-Generated Textcited by 9
  4. 2025A Statistical Case Against Empirical Human-AI Alignmentcited by 6
  5. 2026Beyond Arrow: From Impossibility to Possibilities in Multi-Criteria Benchmarkingcited by 1
  6. 2024Reciprocal Learningcited by 0
  7. 2026Regime-Conditioned Evaluation in Multi-Context Bayesian Optimizationcited by 0
  8. 2026Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantificationcited by 0