mlsanityData to 7 Oct 2026

An Investigation into Value Misalignment in LLM-Generated Texts for Cultural Heritage

arXiv 2501.02039 ↗First posted 3 Jan 2025cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

As Large Language Models (LLMs) become increasingly prevalent in tasks related to cultural heritage, such as generating descriptions of historical monuments, translating ancient texts, preserving oral traditions, and creating educational content, their ability to produce accurate and culturally aligned texts is being increasingly relied upon by users and researchers. However, cultural value misalignments may exist in generated texts, such as the misrepresentation of historical facts, the erosion of cultural identity, and the oversimplification of complex cultural narratives, which may lead to severe consequences. Therefore, investigating value misalignment in the context of LLM for cultural heritage is crucial for mitigating these risks, yet there has been a significant lack of systematic and comprehensive study and investigation in this area. To fill this gap, we systematically assess the reliability of LLMs in generating culturally aligned texts for cultural heritage-related tasks. We conduct a comprehensive evaluation by compiling an extensive set of 1066 query tasks covering 5 widely recognized categories with 17 aspects within the knowledge framework of cultural heritage across 5 open-source LLMs, and examine both the type and rate of cultural value misalignments in the generated texts. Using both automated and manual approaches, we effectively detect and analyze the cultural value misalignments in LLM-generated texts. Our findings are concerning: over 65% of the generated texts exhibit notable cultural misalignments, with certain tasks demonstrating almost complete misalignment with key cultural values. Beyond these findings, this paper introduces a benchmark dataset and a comprehensive evaluation workflow that can serve as a valuable resource for future research aimed at enhancing the cultural sensitivity and reliability of LLMs.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 18 of its 70 references that are in the map. First posted on arXiv 3 Jan 2025.

Builds on

18 of 70 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024A Survey on LLM-as-a-Judgecited by 1,384
  3. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  4. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  5. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  6. 2020Artificial Intelligence, Values and Alignmentcited by 443
  7. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  8. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  9. 2023Human-like Summarization Evaluation with ChatGPTcited by 126
  10. 2023A Survey on Federated Unlearning: Challenges, Methods, and Future Directionscited by 58
  11. 2024How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensionscited by 51
  12. 2024Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learningcited by 18
6 more
  1. 2024Halu-J: Critique-Based Hallucination Judgecited by 14
  2. 2024An Investigation into Misuse of Java Security APIs by Large Language Modelscited by 11
  3. 2024How Well Do LLMs Identify Cultural Unity in Diversity?cited by 8
  4. 2024AI Safety Landscape for Large Language Models: Taxonomy, State-of-the-art, and Future Directionscited by 7
  5. 2024Benchmarking Cognitive Domains for LLMs: Insights from Taiwanese Hakka Culturecited by 6
  6. 2024Exploring Large Language Models on Cross-Cultural Values in Connection with Training Methodologycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifactscited by 10
  2. 2025Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreationcited by 5
  3. 2025Do Large Language Models Truly Understand Cross-cultural Differences?cited by 5
  4. 2026CHisAgent: A Multi-Agent Framework for Event Taxonomy Construction in Ancient Chinese Cultural Systemscited by 1
  5. 2026Attribution Techniques for Mitigating Hallucinated Information in RAG Systems: A Surveycited by 0
  6. 2026Can Large Language Models Make Everyone Happy?cited by 0
  7. 2026Are Aligned Large Language Models Still Misaligned?cited by 0
  8. 2026Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branchescited by 0