# From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge

arXiv: https://arxiv.org/abs/2510.20043 · HTML: https://mlsanity.com/paper/2510.20043/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 11 of 11)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Having Beer after Prayer? Measuring Cultural Bias in Large Language Models](https://mlsanity.com/paper/2305.14456/) (2023)
- [ArabicaQA: A Comprehensive Dataset for Arabic Question Answering](https://mlsanity.com/paper/2403.17848/) (2024)
- [Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models](https://mlsanity.com/paper/2402.13492/) (2024)
- [Automated Literature Review Using NLP Techniques and LLM-Based Retrieval-Augmented Generation](https://mlsanity.com/paper/2411.18583/) (2024)
- [Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs](https://mlsanity.com/paper/2407.00416/) (2024)
- [Do Large Language Models Speak All Languages Equally? A Comparative Study in Low-Resource Settings](https://mlsanity.com/paper/2408.02237/) (2024)
- [Evaluation of Question Answering Systems: Complexity of judging a natural language](https://mlsanity.com/paper/2209.12617/) (2022)
- [A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English](https://mlsanity.com/paper/2011.00960/) (2020)
- [Advancements and Challenges in Bangla Question Answering Models: A Comprehensive Review](https://mlsanity.com/paper/2412.11823/) (2024)

## Built on by (most cited first, 2 of 2)

- [Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models](https://mlsanity.com/paper/2606.00919/) (2026)
- [Analyzing performance of Retrieval Augmented Generation models for Question Answering Tasks](https://mlsanity.com/paper/2610.20419/) (2026)

## Abstract (arXiv)

Recent progress in NLP research has demonstrated remarkable capabilities of large language models (LLMs) across a wide range of tasks. While recent multilingual benchmarks have advanced cultural evaluation for LLMs, critical gaps remain in capturing the nuances of low-resource cultures. Our work addresses these limitations through a Bengali Language Cultural Knowledge (BLanCK) dataset including folk traditions, culinary arts, and regional dialects. Our investigation of several multilingual language models shows that while these models perform well in non-cultural categories, they struggle significantly with cultural knowledge and performance improves substantially across all models when context is provided, emphasizing context-aware architectures and culturally curated training data.