# Capabilities Ain't All You Need: Measuring Propensities in AI

arXiv: https://arxiv.org/abs/2602.18182 · HTML: https://mlsanity.com/paper/2602.18182/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 20 Feb 2026.

## Builds on (most cited first, 25 of 25)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [What Will it Take to Fix Benchmarking in Natural Language Understanding?](https://mlsanity.com/paper/2104.02145/) (2021)
- [Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark](https://mlsanity.com/paper/2304.03279/) (2023)
- [Model evaluation for extreme risks](https://mlsanity.com/paper/2305.15324/) (2023)
- ["I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://mlsanity.com/paper/2205.09209/) (2022)
- [Ministral 3](https://mlsanity.com/paper/2601.08584/) (2026)
- [Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models](https://mlsanity.com/paper/2512.13607/) (2025)
- [Revealing the structure of language model capabilities](https://mlsanity.com/paper/2306.10062/) (2023)
- [A Rosetta Stone for AI Benchmarks](https://mlsanity.com/paper/2512.00193/) (2025)
- [General Scales Unlock AI Evaluation with Explanatory and Predictive Power](https://mlsanity.com/paper/2503.06378/) (2025)
- [Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods](https://mlsanity.com/paper/2505.05541/) (2025)
- [Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culture](https://mlsanity.com/paper/2502.15620/) (2025)

## Built on by (most cited first, 6 of 6)

- [Measuring Progress Toward AGI: A Cognitive Framework](https://mlsanity.com/paper/2605.28405/) (2026)
- [LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs](https://mlsanity.com/paper/2606.06286/) (2026)
- [Item Response Theory for AI Safety](https://mlsanity.com/paper/2608.05086/) (2026)
- [Epistemic Stance Flexibility Probing: Measuring Prompt-Conditioned Register Shift in Large Language Models](https://mlsanity.com/paper/2607.12739/) (2026)
- [Frontier AI Forecasting Has a Measurement Problem: An Audit of Progress Evidence](https://mlsanity.com/paper/2608.14903/) (2026)
- [Using profiles of cognitive capability to assess AI suitability for workplace tasks](https://mlsanity.com/paper/2608.25623/) (2026)

## Abstract (arXiv)

AI evaluation has primarily focused on measuring capabilities, with formal approaches inspired from Item Response Theory (IRT) being increasingly applied. Yet propensities - the tendencies of models to exhibit particular behaviours - play a central role in determining both performance and safety outcomes. However, traditional IRT describes a model's success on a task as a monotonic function of model capabilities and task demands, an approach unsuited to propensities, where both excess and deficiency can be problematic. Here, we introduce the first formal framework for measuring AI propensities by using a bilogistic formulation for model success, which attributes high success probability when the model's propensity is within an "ideal band". Further, we estimate the limits of the ideal band using LLMs equipped with newly developed task-agnostic rubrics. Applying our framework to six families of LLM models whose propensities are incited in either direction, we find that we can measure how much the propensity is shifted and what effect this has on the tasks. Critically, propensities estimated using one benchmark successfully predict behaviour on held-out tasks. Moreover, we obtain stronger predictive power when combining propensities and capabilities than either separately. More broadly, our framework showcases how rigorous propensity measurements can be conducted and how it yields gains over solely using capability evaluations to predict AI behaviour.