mlsanityData to 7 Oct 2026

Dodgersort: Uncertainty-Aware VLM-Guided Human-in-the-Loop Pairwise Ranking

arXiv 2603.20839 ↗First posted 21 Mar 2026cs.CV cs.AI cs.HCcited by 1

Not on the map. Why?

Abstract (arXiv)

Pairwise comparison labeling is emerging as it yields higher inter-rater reliability than conventional classification labeling, but exhaustive comparisons require quadratic cost. We propose Dodgersort, which leverages CLIP-based hierarchical pre-ordering, a neural ranking head and probabilistic ensemble (Elo, BTL, GP), epistemic--aleatoric uncertainty decomposition, and information-theoretic pair selection. It reduces human comparisons while improving the reliability of the rankings. In visual ranking tasks in medical imaging, historical dating, and aesthetics, Dodgersort achieves a 11--16% annotation reduction while improving inter-rater reliability. Cross-domain ablations across four datasets show that neural adaptation and ensemble uncertainty are key to this gain. In FG-NET with ground-truth ages, the framework extracts 5--20× more ranking information per comparison than baselines, yielding Pareto-optimal accuracy--efficiency trade-offs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 8 of its 14 references that are in the map. First posted on arXiv 21 Mar 2026.

Builds on

8 of 14 references.

References · 8

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  3. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 674
  4. 2022K-LITE: Learning Transferable Visual Models with External Knowledgecited by 79
  5. 2023Sorting with Predictionscited by 14
  6. 2024Active Preference Learning for Ordering Items In- and Out-of-samplecited by 6
  7. 2022Decreasing Annotation Burden of Pairwise Comparisons with Human-in-the-Loop Sorting: Application in Medical Image Artifact Ratingcited by 5
  8. 2025EZ-Sort: Efficient Pairwise Comparison via Zero-Shot CLIP-Based Pre-Ordering and Human-in-the-Loop Sortingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Surprise-Guided MergeSort: Budget-Efficient Human-in-the-Loop Ranking via Adaptive Comparison Schedulingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Ranking-aware adapter for text-driven image ordering with CLIP
  2. 2026From Saliency to Discriminability: Rank-Preserving Visual Token Pruning for VLM Rerankers
  3. 2024Ranked from Within: Ranking Large Multimodal Models Without Labels
  4. 2026VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation
  5. 2026Low Rank for Rank: Uncertainty-Aware Task-Specific LLM Ranking under Sparse Pairwise Comparisons
  6. 2026Latent Ordinal Evidence, Misaligned Outputs: Inference-Time Ordinal Lens Alignment for Multimodal LLMs
  7. 2025Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere
  8. 2025Human Uncertainty-Aware Data Selection and Automatic Labeling in Visual Question Answering