# Identity-Link IRT for Label-Free LLM Evaluation: Preserving Additivity in TVD-MI Scores

arXiv: https://arxiv.org/abs/2510.14966 · HTML: https://mlsanity.com/paper/2510.14966/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Builds on (most cited first, 6 of 6)

- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Lost in Benchmarks? Rethinking Large Language Model Benchmarking with Item Response Theory](https://mlsanity.com/paper/2505.15055/) (2025)
- [Leveraging LLM-Respondents for Item Evaluation: a Psychometric Analysis](https://mlsanity.com/paper/2407.10899/) (2024)
- [Crowdsourced Judgement Elicitation with Endogenous Proficiency](https://mlsanity.com/paper/1303.0799/) (2013)
- [Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization](https://mlsanity.com/paper/2409.18433/) (2024)
- [Benchmarking LLMs' Judgments with No Gold Standard](https://mlsanity.com/paper/2411.07127/) (2024)

## Built on by (most cited first, 2 of 2)

- [Can We Trust Item Response Theory for AI Evaluation?](https://mlsanity.com/paper/2607.15190/) (2026)
- [DualEval: Joint Model-Item Calibration for Unified LLM Evaluation](https://mlsanity.com/paper/2606.26429/) (2026)

## Abstract (arXiv)

Pairwise comparisons of large language models using total variation distance mutual information (TVD-MI) produce binary critic decisions per pair. We show that averaging TVD-MI's binary trials yields centered-probability scores with additive structure suitable for item-response theory (IRT) without nonlinear link functions. Maximum-likelihood approaches to IRT use logistic links, but we find empirically that these transformations introduce curvature that breaks additivity: across three domains, the identity link yields median curl on raw data of 0.080-0.150 (P95 = [0.474, 0.580]), whereas probit/logit introduce substantially higher violations (median [0.245, 0.588], P95 [0.825, 2.252]). We derive this clipped-linear model from Gini entropy maximization, yielding a box-constrained least-squares formulation that handles boundary saturation. At 33% coverage, we achieve holdout RMSE 0.117 ± 0.008 while preserving agent rankings (Spearman ρ = 0.972 ± 0.015), three times fewer evaluations than full dense. Judge robustness analysis (GPT-4o-mini vs. Llama3-70b) shows strong agreement in agent rankings (ρ = 0.872) and consistent identity-link advantage. TVD-MI's geometry is best preserved by identity mapping for efficient LLM evaluation, applicable to other bounded-response domains.