# Learning What Evaluators Value: A Reliable Approach to Modeling Evaluator Preferences

arXiv: https://arxiv.org/abs/2605.16615 · HTML: https://mlsanity.com/paper/2605.16615/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 15 May 2026.

## Builds on (most cited first, 13 of 13)

- [Simple, Robust and Optimal Ranking from Pairwise Comparisons](https://mlsanity.com/paper/1512.08949/) (2015)
- [A Voting-Based System for Ethical Decision Making](https://mlsanity.com/paper/1709.06692/) (2017)
- [Adapting a Kidney Exchange Algorithm to Align with Human Values](https://mlsanity.com/paper/2005.09755/) (2020)
- [HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition](https://mlsanity.com/paper/2402.15754/) (2024)
- [Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues](https://mlsanity.com/paper/1510.05610/) (2015)
- [Constrained Monotonic Neural Networks](https://mlsanity.com/paper/2205.11775/) (2022)
- [Is Your Paper Being Reviewed by an LLM? Benchmarking AI Text Detection in Peer Review](https://mlsanity.com/paper/2502.19614/) (2025)
- [HotelRec: a Novel Very Large-Scale Hotel Recommendation Dataset](https://mlsanity.com/paper/2002.06854/) (2020)
- [Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation](https://mlsanity.com/paper/1903.01395/) (2019)
- [Loss Functions, Axioms, and Peer Review](https://mlsanity.com/paper/1808.09057/) (2018)
- [Can AI Model the Complexities of Human Moral Decision-Making? A Qualitative Study of Kidney Allocation Decisions](https://mlsanity.com/paper/2503.00940/) (2025)
- [Bridging Human and LLM Judgments: Understanding and Narrowing the Gap](https://mlsanity.com/paper/2508.12792/) (2025)
- [Feeling the Bern: Adaptive Estimators for Bernoulli Probabilities of Pairwise Comparisons](https://mlsanity.com/paper/1603.06881/) (2016)

## Abstract (arXiv)

In many applications, human and LLM evaluators use assessments of relevant criteria to create an overall evaluation for an item or individual. For example, in admissions, committees assess candidates on attributes such as test scores, GPA, and research experience to evaluate their overall fit for the program. Another example arises in medical care, where clinicians use patient reports of symptoms to consider preliminary diagnoses and assess risks. Each setting involves mapping multiple criteria to an overall evaluation---a process that reflects the evaluator's underlying preferences. We focus on the fundamental question of learning these preferences. Many applications of this problem make specific modeling assumptions on evaluator preferences that may be substantially violated in the real world. We make the minimal assumption that the preference function is coordinate-wise non-decreasing, which is reasonable in a large number of evaluation settings. We theoretically characterize the severity of model mismatch for many common assumptions and show that it can lead to significant issues for learning evaluator preferences and other important downstream tasks. We then present an algorithm for learning evaluators' preferences that is robust to model mismatch. We prove theoretically that our algorithm can learn any preference function without sacrificing performance when the linearity assumption holds. Evaluations of our algorithm with synthetic simulations and real-world data confirm its ability to learn preferences robustly and illustrate key aspects of LLM and human preferences. To conclude, we present a case study of how to use our method to provide insights into reviewer preferences and increase fairness in the peer review process.