mlsanityData to 7 Oct 2026

Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese

arXiv 2402.17302 ↗First posted 27 Feb 2024cs.CLcited by 36

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are increasingly being used to generate synthetic data for training and evaluating models. However, it is unclear whether they can generate a good quality of question answering (QA) dataset that incorporates knowledge and cultural nuance embedded in a language, especially for low-resource languages. In this study, we investigate the effectiveness of using LLMs in generating culturally relevant commonsense QA datasets for Indonesian and Sundanese languages. To do so, we create datasets for these languages using various methods involving both LLMs and human annotators, resulting in ~4.5K questions per language (~9K in total), making our dataset the largest of its kind. Our experiments show that automatic data adaptation from an existing English dataset is less effective for Sundanese. Interestingly, using the direct generation method on the target language, GPT-4 Turbo can generate questions with adequate general knowledge in both languages, albeit not as culturally 'deep' as humans. We also observe a higher occurrence of fluency errors in the Sundanese dataset, highlighting the discrepancy between medium- and lower-resource languages.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 17 of its 51 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

17 of 51 references.

References · 17

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  6. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  7. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  8. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  9. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  10. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  11. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  12. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
5 more
  1. 2022NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languagescited by 67
  2. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 65
  3. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
  4. 2018MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledgecited by 30
  5. 2024Large Malaysian Language Model Based on Mistral for Enhanced Local Language Understandingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 18; 2026: 8.

36 papers.

Sort:

By area:

  1. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 171
  2. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  3. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 101
  4. 2024CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologiescited by 98
  5. 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 38
  6. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  7. 2025CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesiscited by 14
  8. 2024A Survey on Human-Centric LLMscited by 13
  9. 2025Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMscited by 12
  10. 2024mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humanscited by 10
  11. 2025Meta-Cultural Competence: Climbing the Right Hill of Cultural Awarenesscited by 8
  12. 2025SaudiCulture: A Benchmark for Evaluating Large Language Models Cultural Competence within Saudi Arabiacited by 8
24 more
  1. 2024Towards Cross-Lingual Explanation of Artwork in Large-scale Vision Language Modelscited by 7
  2. 2025ScIRGen: Synthesize Realistic and Large-Scale RAG Dataset for Scientific Researchcited by 7
  3. 2025CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesiscited by 6
  4. 2025Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreationcited by 5
  5. 2025Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and Accountabilitycited by 5
  6. 2025What are Foundation Models Cooking in the Post-Soviet World?cited by 3
  7. 2025Can LLMs Grasp Implicit Cultural Values? Benchmarking LLMs' Cultural Intelligence with CQ-Benchcited by 3
  8. 2025Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answeringcited by 3
  9. 2025Synthetic Voice Data for Automatic Speech Recognition in African Languagescited by 3
  10. 2025Preserving Cultural Identity with Context-Aware Translation Through Multi-Agent AI Systemscited by 2
  11. 2025The Unintended Trade-off of AI Alignment:Balancing Hallucination Mitigation and Safety in LLMscited by 2
  12. 2026CALM: Culturally Self-Aware Language Modelscited by 2
  13. 2026Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classificationcited by 2
  14. 2025Finding A Voice: Exploring the Potential of African American Dialect and Voice Generation for Chatbotscited by 1
  15. 2025RoSE: Round-robin Synthetic Data Evaluation for Selecting LLM Generators without Human Test Setscited by 1
  16. 2026No Shortcuts to Culture: Indonesian Multi-hop Question Answering for Complex Cultural Understandingcited by 1
  17. 2024Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustmentscited by 0
  18. 2025NativQA Framework: Enabling LLMs and VLMs with Native, Local, and Everyday Knowledgecited by 0
  19. 2025Ko-PIQA: A Korean Physical Commonsense Reasoning Dataset with Cultural Contextcited by 0
  20. 2026To Generate or Discriminate? Methodological Considerations for Measuring Cultural Alignment in LLMscited by 0
  21. 2026Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMscited by 0
  22. 2026End-to-End Chatbot Evaluation with Adaptive Reasoning and Uncertainty Filteringcited by 0
  23. 2026CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languagescited by 0
  24. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0