mlsanityData to 14 Oct 2026

CARE: Multilingual Human Preference Learning for Cultural Awareness

arXiv 2504.05154 ↗First posted 7 Apr 2025cs.CLcited by 15

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Language Models (LMs) are typically tuned with human preferences to produce helpful responses, but the impact of preference tuning on the ability to handle culturally diverse queries remains understudied. In this paper, we systematically analyze how native human cultural preferences can be incorporated into the preference learning process to train more culturally aware LMs. We introduce CARE, a multilingual resource containing 3,490 culturally specific questions and 31.7k responses with human judgments. We demonstrate how a modest amount of high-quality native preferences improves cultural awareness across various LMs, outperforming larger generic preference data. Our analyses reveal that models with stronger initial cultural performance benefit more from alignment, leading to gaps among models developed in different regions with varying access to culturally relevant data. CARE is publicly available at https://github.com/Guochry/CARE.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 45 of its 70 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

45 of 70 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Self-Pluralising Culture Alignment for Large Language Models16 Oct
  2. 2022Fine-tuning language models to find agreement among humans with diverse preferences28 Nov

Other references · 43

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  9. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  10. 2023LIMA: Less Is More for Alignmentcited by 1,100
  11. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
31 more
  1. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  2. 2018Gender Bias in Coreference Resolutioncited by 442
  3. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  4. 2023Better Zero-Shot Reasoning with Role-Play Promptingcited by 223
  5. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
  6. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 177
  7. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 173
  8. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
  9. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 112
  10. 2024Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilitiescited by 111
  11. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  12. 2025MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluationcited by 79
  13. 2022Extracting Cultural Commonsense Knowledge at Scalecited by 76
  14. 2024The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harmcited by 70
  15. 2024MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimizationcited by 69
  16. 2024Unintended Impacts of LLM Alignment on Global Representationcited by 66
  17. 2024Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsensecited by 51
  18. 2025Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMscited by 39
  19. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  20. 2024Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledgecited by 29
  21. 2024CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledgecited by 28
  22. 2024Language Imbalance Driven Rewarding for Multilingual Self-improvingcited by 28
  23. 2024LLM-GLOBE: A Benchmark Evaluating the Cultural Values Embedded in LLM Outputcited by 27
  24. 2025On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomenacited by 22
  25. 2024JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluationcited by 19
  26. 2024Cultural Palette: Pluralising Culture Alignment via Multi-agent Palettecited by 16
  27. 2025Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMscited by 15
  28. 2024RENOVI: A Benchmark Towards Remediating Norm Violations in Socio-Cultural Conversationscited by 14
  29. 2024CamelEval: Advancing Culturally Aligned Arabic Language Models and Benchmarkscited by 10
  30. 2025More is Less: The Pitfalls of Multi-Model Synthetic Preference Data in DPO Safety Alignmentcited by 5
  31. 2024How Chinese are Chinese Language Models? The Puzzling Lack of Language Policy in China's LLMscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 12.

15 papers.

Sort:

By area:

  1. 2026From RAG to Agentic RAG for Faithful Islamic Question Answeringcited by 9
  2. 2025Multimodal Cultural Safety: Evaluation Framework and Alignment Strategiescited by 8
  3. 2026Tears or Cheers? Benchmarking LLMs via Culturally Elicited Distinct Affective Responsescited by 6
  4. 2025Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignmentcited by 2
  5. 2025Camellia: Benchmarking Cultural Biases in LLMs for Asian Languagescited by 2
  6. 2026Scaling Participation in Modular AI Systemscited by 2
  7. 2026Learning to Route Languages for Multilingual Policy Optimizationcited by 1
  8. 2026PLURAL: A Global Dataset for Value Alignmentcited by 1
  9. 2026From National Curricula to Cultural Awareness: Constructing Open-Ended Culture-Specific Question Answering Datasetcited by 0
  10. 2026Toward Culturally Grounded Natural Language Processingcited by 0
  11. 2026Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contextscited by 0
  12. 2026CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languagescited by 0
3 more
  1. 2026CoCoA: Context-Conditional Cultural Alignment for Large Language Modelscited by 0
  2. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
  3. 2026FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026DiSCo: A Distribution-First Steering and Cultural Prior Evaluation Framework for Measuring Cultural Preference Bias in LLMs
  2. 2025Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Dataset
  3. 2024A Survey on Human Preference Learning for Large Language Models
  4. 2025CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis
  5. 2025CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimization
  6. 2026Scenario-based Probing and Steering Cultural Values in Large Language Models--Extended Version
  7. 2024Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models
  8. 2026Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs