CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languages
Not on the map. Why?
Abstract (arXiv)
Culture is lived through conversation, yet existing Indonesian cultural commonsense benchmarks evaluate LLMs on short and isolated prompts, stripping away the dialogic context in which cultural nuances actually surface. We introduce CultureTalk-ID, the first dialogue-based benchmark for cultural commonsense in Indonesian and its local languages, comprising 4,496 culturally grounded dialogues across 11 languages and 13 culturally salient topics, curated through a multi-stage human pipeline with native speakers to ensure authenticity. CultureTalk-ID introduces three complementary tasks, namely dialogue-based multiple-choice cultural commonsense reasoning, culturally faithful machine translation, and language steering, which jointly probe whether LLMs can understand, transfer, and generate culturally grounded language.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 38 references that are in the map. First posted on arXiv 23 Jul 2026.
Builds on
20 of 38 references.
References · 20
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
- 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
- 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
- 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
- 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 171
- 2022NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languagescited by 67
- 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 65
- 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 38
- 2024Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanesecited by 36
- 2023COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuancescited by 35
8 more
- 2021IndoNLI: A Natural Language Inference Dataset for Indonesiancited by 24
- 2022Cross-Lingual Dialogue Dataset Creation via Outline-Based Generationcited by 23
- 2025CARE: Multilingual Human Preference Learning for Cultural Awarenesscited by 15
- 2022MDIA: A Benchmark for Multilingual Dialogue Generation in 46 Languagescited by 13
- 2020Cross-Lingual Machine Speech Chain for Javanese, Sundanese, Balinese, and Bataks Speech Recognition and Synthesiscited by 8
- 2025LoraxBench: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languagescited by 5
- 2025SEADialogues: A Multilingual Culturally Grounded Multi-turn Dialogue Dataset on Southeast Asian Languagescited by 3
- 2026No Shortcuts to Culture: Indonesian Multi-hop Question Answering for Complex Cultural Understandingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues
- 2025MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
- 2025Culturally-Nuanced Story Generation for Reasoning in Low-Resource Languages: The Case of Javanese and Sundanese
- 2026SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding
- 2026Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency
- 2025CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis
- 2026XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triad
- 2025CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesis