# Review on Ranking and Selection: A New Perspective

arXiv: https://arxiv.org/abs/2008.00249 · HTML: https://mlsanity.com/paper/2008.00249/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2020.

## Built on by (most cited first, 28 of 28)

- [Top Two Algorithms Revisited](https://mlsanity.com/paper/2206.05979/) (2022)
- [Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian](https://mlsanity.com/paper/2211.00716/) (2022)
- [Minimax Optimal Algorithms for Fixed-Budget Best Arm Identification](https://mlsanity.com/paper/2206.04646/) (2022)
- [Rate-optimal Bayesian Simple Regret in Best Arm Identification](https://mlsanity.com/paper/2111.09885/) (2021)
- [Non-Asymptotic Analysis of a UCB-based Top Two Algorithm](https://mlsanity.com/paper/2210.05431/) (2022)
- [Best Arm Identification with Fairness Constraints on Subpopulations](https://mlsanity.com/paper/2304.04091/) (2023)
- [SPRT-based Efficient Best Arm Identification in Stochastic Bandits](https://mlsanity.com/paper/2207.11158/) (2022)
- [Worst-Case Optimal Multi-Armed Gaussian Best Arm Identification with a Fixed Budget](https://mlsanity.com/paper/2310.19788/) (2023)
- [Collaborative Intelligence in Sequential Experiments: A Human-in-the-Loop Framework for Drug Discovery](https://mlsanity.com/paper/2405.03942/) (2024)
- [Genetic multi-armed bandits: a reinforcement learning approach for discrete optimization via simulation](https://mlsanity.com/paper/2302.07695/) (2023)
- [Sample-Efficient "Clustering and Conquer" Procedures for Parallel Large-Scale Ranking and Selection](https://mlsanity.com/paper/2402.02196/) (2024)
- [Suboptimal Performance of the Bayes Optimal Algorithm in Frequentist Best Arm Identification](https://mlsanity.com/paper/2202.05193/) (2022)
- [Choosing Answers in varepsilon-Best-Answer Identification for Linear Bandits](https://mlsanity.com/paper/2206.04456/) (2022)
- [Identifying the Best Arm in the Presence of Global Environment Shifts](https://mlsanity.com/paper/2408.12581/) (2024)
- [Admissibility of Completely Randomized Trials: A Large-Deviation Approach](https://mlsanity.com/paper/2506.05329/) (2025)
- [SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations](https://mlsanity.com/paper/2511.00685/) (2025)
- [TESO Tabu Enhanced Simulation Optimization for Noisy Black Box Problems](https://mlsanity.com/paper/2512.24007/) (2025)
- [Best-Arm Identification with Generative Proxy](https://mlsanity.com/paper/2607.06879/) (2026)
- [Selecting the Best Optimizing System](https://mlsanity.com/paper/2201.03065/) (2022)
- [Efficient Learning for Selecting Top-m Context-Dependent Designs](https://mlsanity.com/paper/2305.04086/) (2023)
- [Fixed Confidence Best Arm Identification in the Bayesian Setting](https://mlsanity.com/paper/2402.10429/) (2024)
- [New Additive OCBA Procedures for Robust Ranking and Selection](https://mlsanity.com/paper/2412.06020/) (2024)
- [Regular Tree Search for Simulation Optimization](https://mlsanity.com/paper/2506.17696/) (2025)
- [Sequential Additivity in Distributionally Robust Ranking and Selection](https://mlsanity.com/paper/2509.06147/) (2025)
- [UCB for Large-Scale Pure Exploration: Beyond Sub-Gaussianity](https://mlsanity.com/paper/2511.22273/) (2025)
- [Depth over Fidelity in Fixed-Budget Noisy Evolution Strategies](https://mlsanity.com/paper/2606.06555/) (2026)
- [Annealed Entropic Allocation for Ranking and Selection](https://mlsanity.com/paper/2606.11347/) (2026)
- [Ranking-and-Selection with Multiple Correct Answers and Non-Answerable Estimates](https://mlsanity.com/paper/2606.21889/) (2026)

## Abstract (arXiv)

In this paper, we briefly review the development of ranking-and-selection (R&S) in the past 70 years, especially the theoretical achievements and practical applications in the last 20 years. Different from the frequentist and Bayesian classifications adopted by Kim and Nelson (2006b) and Chick (2006) in their review articles, we categorize existing R&S procedures into fixed-precision and fixed-budget procedures, as in Hunter and Nelson (2017). We show that these two categories of procedures essentially differ in the underlying methodological formulations, i.e., they are built on hypothesis testing and dynamic-programming, respectively. In light of this variation, we review in detail some well-known procedures in the literature and show how they fit into these two formulations. In addition, we discuss the use of R&S procedures in solving various practical problems and propose what we think are the important research questions in the field.