mlsanityData to 7 Oct 2026

SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding

arXiv 2606.03284 ↗First posted 2 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Frontier LLMs perform well in Western contexts, but remain poorly tested on underrepresented cultures such as those in Southeast Asia (SEA). Existing NLI benchmarks are largely Western-centric, translation-derived, or monolingual, limiting their ability to measure culturally grounded reasoning. We introduce SEA-NLI, a native, culturally grounded NLI benchmark covering eight SEA countries in English and native regional languages, verified by native speakers. Across 17 encoder and decoder models, we observe a low performance from all models, especially for knowledge-intensive categories such as Languages and Science and Technology. Our analysis shows that failure cases mainly stem from missing SEA cultural knowledge: SEA-adapted models and culture-aware prompting improve performance, while CoT prompting offers limited gains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

33 of 49 references.

References · 33

  1. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  2. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  3. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  4. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  5. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  6. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  7. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  8. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  9. 2024Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluationcited by 181
  10. 2024Investigating Cultural Alignment of Large Language Modelscited by 140
  11. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmarkcited by 118
  12. 2020OCNLI: Original Chinese Natural Language Inferencecited by 86
21 more
  1. 2022Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasetscited by 54
  2. 2020KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understandingcited by 37
  3. 2020Towards Robustifying NLI Models Against Lexical Dataset Biasescited by 34
  4. 2019On Adversarial Removal of Hypothesis-only Bias in Natural Language Inferencecited by 29
  5. 2021IndoNLI: A Natural Language Inference Dataset for Indonesiancited by 24
  6. 2021SICKNL: A Dataset for Dutch Natural Language Inferencecited by 20
  7. 2025SEA-HELM: Southeast Asian Holistic Evaluation of Language Modelscited by 17
  8. 2025SEA-LION: Southeast Asian Languages in One Networkcited by 17
  9. 2024A synthetic data approach for domain generalization of NLI modelscited by 15
  10. 2022Compositional Evaluation on Japanese Textual Entailment and Similaritycited by 14
  11. 2025Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lenscited by 13
  12. 2024Translation Errors Significantly Impact Low-Resource Languages in Cross-Lingual Learningcited by 9
  13. 2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asiacited by 8
  14. 2025Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmarcited by 8
  15. 2025Seeing Culture: A Benchmark for Visual Reasoning and Groundingcited by 8
  16. 2023Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentationcited by 7
  17. 2025CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesiscited by 6
  18. 2024A New Benchmark Dataset and Mixture-of-Experts Language Models for Adversarial Natural Language Inference in Vietnamesecited by 4
  19. 2025Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selectioncited by 4
  20. 2026XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triadcited by 2
  21. 2026FLANS at SemEval-2026 Task 7: RAG with Open-Sourced Smaller LLMs for Everyday Knowledge Across Diverse Languages and Culturescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SEA-SpeechBench: A Large-Scale Multitask Benchmark for Speech Understanding Across Southeast Asia
  2. 2025Culturally-Grounded Chain-of-Thought (CG-CoT):Enhancing LLM Performance on Culturally-Specific Tasks in Low-Resource Languages
  3. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Models
  4. 2025MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation
  5. 2025Culturally-Nuanced Story Generation for Reasoning in Low-Resource Languages: The Case of Javanese and Sundanese
  6. 2026SEABED: SouthEast Asian Benchmark for Evaluating Audio Reasoning
  7. 2026CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languages
  8. 2025LingBench++: A Linguistically-Informed Benchmark and Reasoning Framework for Multi-Step and Cross-Cultural Inference with LLMs