mlsanityData to 7 Oct 2026

Language Models as Measurement Apparatus for Culture

arXiv 2607.02459 ↗First posted 2 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Language models are increasingly used to quantify cultural phenomena, but what makes such measurement distinctively cultural? This paper argues that NLP work on culture is a material-discursive practice: the apparatus -- model, data, annotation, evaluation -- participates in constituting the cultural reality it measures, rather than passively recording it. Drawing on Karen Barad's concept of the agential cut -- the contingent boundary between phenomenon and instrument -- I show that the apparatus's substantive design choices draw such boundaries, and that the boundary is entangled from the start because language models have already internalized much of the cultural material they measure. I illustrate this through three case studies on television and film dialogue (measuring structure, interaction, and deviation) and three examinations of the apparatus itself (erasure of cultural markers, attunement to historical material, and agency in an agentic workflow). This big picture analysis proposes a research program that is theory-driven, empirically rigorous, and culturally contingent, treating each agential cut as a conscious commitment, at once methodological and ethical.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 69 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

15 of 69 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Longformer: The Long-Document Transformercited by 3,362
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  4. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  5. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  6. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
  7. 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
  8. 2025Position: Evaluating Generative AI Systems Is a Social Science Measurement Challengecited by 61
  9. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  10. 2022Discovering Differences in the Representation of People using Contextualized Semantic Axescited by 12
  11. 2025Meaning Is Not A Metric: Using LLMs to make cultural context legible at scalecited by 10
  12. 2024On Classification with Large Language Models in Cultural Analyticscited by 9
3 more
  1. 2021Findings on Conversation Disentanglementcited by 6
  2. 2024Subversive Characters and Stereotyping Readers: Characterizing Queer Relationalities with Dialogue-Based Relation Extractioncited by 2
  3. 2025What is a protest anyway? Codebook conceptualization is still a first-order concern in LLM-era classificationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From annotation to reasoning: Culture in language models
  2. 2025The American Ghost in the Machine: How language models align culturally and the effects of cultural prompting
  3. 2025'Too much alignment; not enough culture': Re-balancing cultural alignment practices in LLMs
  4. 2025Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens
  5. 2025Culture is Everywhere: A Call for Intentionally Cultural Evaluation
  6. 2023CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models
  7. 2026Beyond Instrumental and Substitutive Paradigms: Introducing Machine Culture as an Emergent Phenomenon in Large Language Models
  8. 2025Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs