# Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge

arXiv: https://arxiv.org/abs/2502.00561 · HTML: https://mlsanity.com/paper/2502.00561/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2025.

## Built on by (most cited first, 30 of 63)

- [Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs](https://mlsanity.com/paper/2502.01926/) (2025)
- [The Case for "Thick Evaluations" of Cultural Representation in AI](https://mlsanity.com/paper/2503.19075/) (2025)
- [Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming](https://mlsanity.com/paper/2601.18076/) (2026)
- [The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behavior](https://mlsanity.com/paper/2509.19364/) (2025)
- [Taxonomizing Representational Harms using Speech Act Theory](https://mlsanity.com/paper/2504.00928/) (2025)
- [Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor](https://mlsanity.com/paper/2506.14652/) (2025)
- [Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence](https://mlsanity.com/paper/2510.01395/) (2025)
- [ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agents](https://mlsanity.com/paper/2605.20251/) (2026)
- [Social Bias in Popular Question-Answering Benchmarks](https://mlsanity.com/paper/2505.15553/) (2025)
- [Risk Management for Mitigating Benchmark Failure Modes: BenchRisk](https://mlsanity.com/paper/2510.21460/) (2025)
- [Agent Benchmarks Fail Public Sector Requirements](https://mlsanity.com/paper/2601.20617/) (2026)
- [Overseeing Agents Without Constant Oversight: Challenges and Opportunities](https://mlsanity.com/paper/2602.16844/) (2026)
- [Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering](https://mlsanity.com/paper/2606.17799/) (2026)
- [We Need to Measure Data Diversity in NLP -- Better and Broader](https://mlsanity.com/paper/2505.20264/) (2025)
- [Reasoning Beyond Labels: Measuring LLM Sentiment in Low-Resource, Culturally Nuanced Contexts](https://mlsanity.com/paper/2508.04199/) (2025)
- [What is a protest anyway? Codebook conceptualization is still a first-order concern in LLM-era classification](https://mlsanity.com/paper/2510.03541/) (2025)
- [CIRCLE: A Framework for Evaluating AI from a Real-World Lens](https://mlsanity.com/paper/2602.24055/) (2026)
- [From Ground Truth to Measurement: A Statistical Framework for Human Labeling](https://mlsanity.com/paper/2604.07591/) (2026)
- [AI as a Tool for Simulation-Based Experiments in Literary Studies](https://mlsanity.com/paper/2606.02293/) (2026)
- [Offline Preference-Based Trajectory Evaluation](https://mlsanity.com/paper/2606.17541/) (2026)
- [SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testing](https://mlsanity.com/paper/2603.01630/) (2026)
- [Making AI Evaluation Deployment Relevant Through Context Specification](https://mlsanity.com/paper/2603.06811/) (2026)
- [BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validity](https://mlsanity.com/paper/2603.18019/) (2026)
- [From Use to Oversight: How Mental Models Influence User Behavior and Output in AI Writing Assistants](https://mlsanity.com/paper/2604.05166/) (2026)
- [Diversity Without Fidelity: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation Simulation](https://mlsanity.com/paper/2604.11840/) (2026)
- [Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructure](https://mlsanity.com/paper/2604.25580/) (2026)
- [Instrument Effects in Language-Model Honesty Evaluation: An Auditable Single-System Demonstration](https://mlsanity.com/paper/2607.14399/) (2026)
- [What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)](https://mlsanity.com/paper/2608.06202/) (2026)
- [Do Assessment Instruments Measure the Same Thing for Humans and LLMs? A Latent Structure Analysis](https://mlsanity.com/paper/2608.15630/) (2026)
- [Rating the Raters: Rasch Measurement Theory for LLM Evaluation](https://mlsanity.com/paper/2608.27463/) (2026)

## Abstract (arXiv)

The measurement tasks involved in evaluating generative AI (GenAI) systems lack sufficient scientific rigor, leading to what has been described as "a tangle of sloppy tests [and] apples-to-oranges comparisons" (Roose, 2024). In this position paper, we argue that the ML community would benefit from learning from and drawing on the social sciences when developing and using measurement instruments for evaluating GenAI systems. Specifically, our position is that evaluating GenAI systems is a social science measurement challenge. We present a four-level framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, behaviors, and impacts of GenAI systems. This framework has two important implications: First, it can broaden the expertise involved in evaluating GenAI systems by enabling stakeholders with different perspectives to participate in conceptual debates. Second, it brings rigor to both conceptual and operational debates by offering a set of lenses for interrogating validity.