# From Human-Level AI Tales to AI Leveling Human Scales

arXiv: https://arxiv.org/abs/2602.18911 · HTML: https://mlsanity.com/paper/2602.18911/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 21 Feb 2026.

## Builds on (most cited first, 10 of 10)

- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Evaluating Large Language Models in Theory of Mind Tasks](https://mlsanity.com/paper/2302.02083/) (2023)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge](https://mlsanity.com/paper/2506.21506/) (2025)
- [Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation](https://mlsanity.com/paper/2502.06559/) (2025)
- [Revealing the structure of language model capabilities](https://mlsanity.com/paper/2306.10062/) (2023)
- [General Scales Unlock AI Evaluation with Explanatory and Predictive Power](https://mlsanity.com/paper/2503.06378/) (2025)
- [Does GPT-4 surpass human performance in linguistic pragmatics?](https://mlsanity.com/paper/2312.09545/) (2023)
- [Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culture](https://mlsanity.com/paper/2502.15620/) (2025)
- [A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Course](https://mlsanity.com/paper/2403.16977/) (2024)

## Built on by (most cited first, 1 of 1)

- [Using profiles of cognitive capability to assess AI suitability for workplace tasks](https://mlsanity.com/paper/2608.25623/) (2026)

## Abstract (arXiv)

Comparing AI models to "human level" is often misleading when benchmark scores are incommensurate or human baselines are drawn from a narrow population. To address this, we propose a framework that calibrates items against the 'world population' and report performance on a common, human-anchored scale. Concretely, we build on a set of multi-level scales for different capabilities where each level should represent a probability of success of the whole world population on a logarithmic scale with a base B. We calibrate each scale for each capability (reasoning, comprehension, knowledge, volume, etc.) by compiling publicly released human test data spanning education and reasoning benchmarks (PISA, TIMSS, ICAR, UKBioBank, and ReliabilityBench). The base B is estimated by extrapolating between samples with two demographic profiles using LLMs, with the hypothesis that they condense rich information about human populations. We evaluate the quality of different mappings using group slicing and post-stratification. The new techniques allow for the recalibration and standardization of scales relative to the whole-world population.