mlsanityData to 14 Oct 2026

CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia

arXiv 2608.28405 ↗First posted 28 Aug 2026cs.CL cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking practical help over multiple turns in culturally grounded scenarios. We introduce CultureConverse, a scalable, multilingual simulation and evaluation harness for culturally grounded assistant dialogue that covers 10 East and Southeast Asian regions, 58 subgroup identities, and 7 domains. Each simulated and evaluated episode produces a scored interaction where the assistant assists the user and infers cultural constraints from partial information. The resulting CultureConverse-DS dataset contains 14,610 benchmark (evaluation) episodes and 274,295 oracle-guided (gold-mode) dialogues. In our benchmark evaluation of 18 models, GPT-5 mini achieves the highest assistance quality. Human annotation experiments suggest that our evaluation framework is a sufficient proxy for human judgment. Performance gains from fine-tuning on 27,860 high-quality CultureConverse-DS samples improve in-domain assistance and transfer out-of-domain to cultural MCQ and safety classification benchmarks. We release the harness, both splits, and judge prompts to support interactive evaluation of cultural competency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 105 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

28 of 105 references.

References · 28

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  5. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  6. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  9. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  10. 2024Quantifying the Persona Effect in LLM Simulationscited by 169
  11. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 144
  12. 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 102
16 more
  1. 2024Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languagescited by 64
  2. 2025Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactionscited by 52
  3. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countriescited by 42
  4. 2023Typhoon: Thai Large Language Modelscited by 30
  5. 2025SEA-LION: Southeast Asian Languages in One Networkcited by 22
  6. 2025SEA-HELM: Southeast Asian Holistic Evaluation of Language Modelscited by 19
  7. 2025Value Compass Benchmarks: A Platform for Fundamental and Validated Evaluation of LLMs Valuescited by 7
  8. 2025CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimizationcited by 6
  9. 2025LoraxBench: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languagescited by 6
  10. 2025Cultural Alignment in Large Language Models Using Soft Prompt Tuningcited by 5
  11. 2025Do Large Language Models Understand Morality Across Cultures?cited by 5
  12. 2025MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluationcited by 5
  13. 2026Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignmentcited by 5
  14. 2025SEADialogues: A Multilingual Culturally Grounded Multi-turn Dialogue Dataset on Southeast Asian Languagescited by 4
  15. 2026AlignCultura: Towards Culturally Aligned Large Language Models?cited by 2
  16. 2026LiveCultureBench: a Multi-Agent, Multi-Cultural Benchmark for Large Language Models in Dynamic Social Simulationscited by 1

Built on by

Not cited yet.