# Coordinates of Capability: A Unified MTMM-Geometric Framework for LLM Evaluation

arXiv: https://arxiv.org/abs/2605.08522 · HTML: https://mlsanity.com/paper/2605.08522/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 8 May 2026.

## Builds on (most cited first, 19 of 19)

- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Large Language Models are Inconsistent and Biased Evaluators](https://mlsanity.com/paper/2405.01724/) (2024)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency](https://mlsanity.com/paper/2502.09621/) (2025)
- [Measuring what Matters: Construct Validity in Large Language Model Benchmarks](https://mlsanity.com/paper/2511.04703/) (2025)
- [Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks](https://mlsanity.com/paper/2310.13800/) (2023)
- [Social Bias Evaluation for Large Language Models Requires Prompt Variations](https://mlsanity.com/paper/2407.03129/) (2024)
- [Shared Global and Local Geometry of Language Model Embeddings](https://mlsanity.com/paper/2503.21073/) (2025)
- [A Survey on Large Language Model Benchmarks](https://mlsanity.com/paper/2508.15361/) (2025)
- [Sensitivity and Robustness of Large Language Models to Prompt Template in Japanese Text Classification Tasks](https://mlsanity.com/paper/2305.08714/) (2023)
- [Drift No More? Context Equilibria in Multi-Turn LLM Interactions](https://mlsanity.com/paper/2510.07777/) (2025)
- [Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas](https://mlsanity.com/paper/2412.14843/) (2024)
- [BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation](https://mlsanity.com/paper/2602.09383/) (2026)
- [Time-To-Inconsistency: A Survival Analysis of Large Language Model Robustness to Adversarial Attacks](https://mlsanity.com/paper/2510.02712/) (2025)
- [Evaluating Performance Drift from Model Switching in Multi-Turn LLM Systems](https://mlsanity.com/paper/2603.03111/) (2026)
- [Visualizing LLM Latent Space Geometry Through Dimensionality Reduction](https://mlsanity.com/paper/2511.21594/) (2025)
- [Quantifying construct validity in large language model evaluations](https://mlsanity.com/paper/2602.15532/) (2026)
- [Large Language Models Can Be a Viable Substitute for Expert Political Surveys When a Shock Disrupts Traditional Measurement Approaches](https://mlsanity.com/paper/2506.06540/) (2025)

## Abstract (arXiv)

The evaluation of Large Language Models (LLMs) faces a critical challenge in construct validity, where fragmented benchmarks and ad hoc metrics frequently conflate method variance, such as prompt sensitivity, with true latent capabilities. Concurrently, emerging research suggests that LLM capabilities and outputs can be modeled as continuous geometric manifolds. In this Systematization of Knowledge (SoK), we bridge these paradigms by proposing a generalized Multi-Trait Multi-Method (MTMM) framework for LLM evaluation. We formalize and unify nine evaluation metrics, including Paraphrase Instability, Drift Score, Overton Width, and Pluralism Score, interpreting them not as isolated scalar values but as geometric measurements within a shared latent coordinate space. This spatial unification factorizes model behavior into three orthogonal latent dimensions: (1) Instability and Sensitivity, (2) Position and Alignment, and (3) Coverage and Expressiveness. By systematically separating task-irrelevant perturbations from true capability spans, the framework provides a theoretically grounded and domain-agnostic taxonomy for robust and empirically stable benchmark design.