mlsanityData to 7 Oct 2026

Computational Hermeneutics: Evaluating generative AI as a cultural technology

arXiv 2604.16403 ↗First posted 31 Mar 2026cs.AI cs.CYcited by 7

Not on the map. Why?

Abstract (arXiv)

Generative AI systems are increasingly recognized as cultural technologies, yet current evaluation frameworks often treat culture as a variable to be measured rather than fundamental to the system's operation. Drawing on hermeneutic theory from the humanities, we argue that GenAI systems function as "context machines" that must inherently address three interpretive challenges: situatedness (meaning only emerges in context), plurality (multiple valid interpretations coexist), and ambiguity (interpretations naturally conflict). We present computational hermeneutics as an emerging framework offering an interpretive account of what GenAI systems do, and how they might do it better. We offer three principles for hermeneutic evaluation -- that benchmarks should be iterative, not one-off; include people, not just machines; and measure cultural context, not just model output. This perspective offers a nascent paradigm for designing and evaluating contemporary AI systems: shifting from standardized questions about accuracy to contextual ones about meaning.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 63 of its 119 references that are in the map. First posted on arXiv 31 Mar 2026.

Builds on

63 of 119 references.

References · 63

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  6. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  7. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  8. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  9. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  10. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  11. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 247
  12. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
38 more
  1. 2024AI Agents That Mattercited by 199
  2. 2023Art or Artifice? Large Language Models and the False Promise of Creativitycited by 168
  3. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  4. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  5. 2024When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboardscited by 134
  6. 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 112
  7. 2022AUC Maximization in the Era of Big Data and AI: A Surveycited by 100
  8. 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
  9. 2023The Metacognitive Demands and Opportunities of Generative AIcited by 85
  10. 2024Building Machines that Learn and Think with Peoplecited by 73
  11. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  12. 2022Mapping global dynamics of benchmark creation and saturation in artificial intelligencecited by 71
  13. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
  14. 2024A Computational Framework for Behavioral Assessment of LLM Therapistscited by 61
  15. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
  16. 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gapcited by 56
  17. 2024Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?cited by 56
  18. 2023Machine Culturecited by 55
  19. 2023Facilitating Self-Guided Mental Health Interventions Through Human-Language Model Interaction: A Case Study of Cognitive Restructuringcited by 53
  20. 2020Targeting the Benchmark: On Methodology in Current Natural Language Processing Researchcited by 44
  21. 2020Bringing the People Back In: Contesting Benchmark Machine Learning Datasetscited by 43
  22. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  23. 2024Assessing and Understanding Creativity in Large Language Modelscited by 35
  24. 2024Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statementscited by 33
  25. 2020Utility is in the Eye of the User: A Critique of NLP Leaderboardscited by 28
  26. 2024Towards interactive evaluations for interaction harms in human-AI systemscited by 27
  27. 2024A theory of appropriateness with applications to generative artificial intelligencecited by 25
  28. 2025Multi-turn Evaluation of Anthropomorphic Behaviours in Large Language Modelscited by 25
  29. 2024SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypescited by 24
  30. 2021Learning to Draw: Emergent Communication through Sketchingcited by 23
  31. 2024How Culture Shapes What People Want From AIcited by 23
  32. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Modelscited by 19
  33. 2022Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionalscited by 16
  34. 2025A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologiescited by 14
  35. 2025CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metricscited by 14
  36. 2024STAR: SocioTechnical Approach to Red Teaming Language Modelscited by 13
  37. 2024From Protoscience to Epistemic Monoculture: How Benchmarking Set the Stage for the Deep Learning Revolutioncited by 12
  38. 2024Contextualized Evaluations: Judging Language Model Responses to Underspecified Queriescited by 12

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026QOuLiPo: What a quantum computer sees when it reads a bookcited by 0
  2. 2026Does Persona Make LLMs K-pop Fans? A Pilot Study of LLM-Based Online Concert Audience Agentscited by 0
  3. 2026World Wide Models: Literary Tools for Cultural AIcited by 0
  4. 2026TimeCapsule: Generative Hallucination as a Method for Historical Sensemakingcited by 0
  5. 2026Where Models Converge and Humans Diverge: A Coverage Framework for Distributional Pluralism in Open-Ended Generationcited by 0
  6. 2026The House with a Million Windows: Interactive Fiction for Narrative Restoryingcited by 0
  7. 2026Reciprocal Collaboration: how lessons from convergence in GLAMs can enhance interdisciplinary AI researchcited by 0