# Hybrid-MST: A Hybrid Active Sampling Strategy for Pairwise Preference Aggregation

arXiv: https://arxiv.org/abs/1810.08851 · HTML: https://mlsanity.com/paper/1810.08851/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2018.

## Built on by (most cited first, 10 of 10)

- [Towards Open-ended Visual Quality Comparison](https://mlsanity.com/paper/2402.16641/) (2024)
- [Subjective Image Quality Assessment with Boosted Triplet Comparisons](https://mlsanity.com/paper/2108.00201/) (2021)
- [Subjective and Objective Quality Assessment of Mobile Gaming Video](https://mlsanity.com/paper/2103.05099/) (2021)
- [Decreasing Annotation Burden of Pairwise Comparisons with Human-in-the-Loop Sorting: Application in Medical Image Artifact Rating](https://mlsanity.com/paper/2202.04823/) (2022)
- [Ranking with Confidence for Large Scale Comparison Data](https://mlsanity.com/paper/2202.01670/) (2022)
- [EZ-Sort: Efficient Pairwise Comparison via Zero-Shot CLIP-Based Pre-Ordering and Human-in-the-Loop Sorting](https://mlsanity.com/paper/2508.21550/) (2025)
- [Highly Efficient No-reference 4K Video Quality Assessment with Full-Pixel Covering Sampling and Training Strategy](https://mlsanity.com/paper/2407.20766/) (2024)
- [E3mo-Bench: A Scalable Benchmark for Multimodal Evoked and Expressed Emotion Understanding via Bayesian Pairwise Alignment](https://mlsanity.com/paper/2608.10796/) (2026)
- [Improving Maximum Likelihood Difference Scaling method to measure inter content scale](https://mlsanity.com/paper/2203.13186/) (2022)
- [Aggregate-then-Calibrate for Human-centered Assessment with Theoretical Guarantees](https://mlsanity.com/paper/2608.02455/) (2026)

## Abstract (arXiv)

In this paper we present a hybrid active sampling strategy for pairwise preference aggregation, which aims at recovering the underlying rating of the test candidates from sparse and noisy pairwise labelling. Our method employs Bayesian optimization framework and Bradley-Terry model to construct the utility function, then to obtain the Expected Information Gain (EIG) of each pair. For computational efficiency, Gaussian-Hermite quadrature is used for estimation of EIG. In this work, a hybrid active sampling strategy is proposed, either using Global Maximum (GM) EIG sampling or Minimum Spanning Tree (MST) sampling in each trial, which is determined by the test budget. The proposed method has been validated on both simulated and real-world datasets, where it shows higher preference aggregation ability than the state-of-the-art methods.