mlsanityData to 7 Oct 2026

BengaliFig: A Low-Resource Challenge for Figurative and Culturally Grounded Reasoning in Bengali

arXiv 2511.20399 ↗First posted 25 Nov 2025cs.CL cs.AIcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models excel on broad multilingual benchmarks but remain to be evaluated extensively in figurative and culturally grounded reasoning, especially in low-resource contexts. We present BengaliFig, a compact yet richly annotated challenge set that targets this gap in Bengali, a widely spoken low-resourced language. The dataset contains 435 unique riddles drawn from Bengali oral and literary traditions. Each item is annotated along five orthogonal dimensions capturing reasoning type, trap type, cultural depth, answer category, and difficulty, and is automatically converted to multiple-choice format through a constraint-aware, AI-assisted pipeline. We evaluate eight frontier LLMs from major providers under zero-shot and few-shot chain-of-thought prompting, revealing consistent weaknesses in metaphorical and culturally specific reasoning. BengaliFig thus contributes both a diagnostic probe for evaluating LLM robustness in low-resource cultural contexts and a step toward inclusive and heritage-aware NLP evaluation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 12 of its 26 references that are in the map. First posted on arXiv 25 Nov 2025.

Builds on

12 of 26 references.

References · 12

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  7. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  8. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  9. 2018Stress Test Evaluation for Natural Language Inferencecited by 211
  10. 2022BanglaNLG and BanglaT5: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Banglacited by 40
  11. 2021BiRdQA: A Bilingual Dataset for Question Answering on Tricky Riddlescited by 10
  12. 2022CC-Riddle: A Question Answering Dataset of Chinese Character Riddlescited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025BnMMLU: Measuring Massive Multitask Language Understanding in Bengali
  2. 2025The Riddle of Reflection: Evaluating Reasoning and Self-Awareness in Multilingual LLMs using Indian Riddles
  3. 2025Leveraging Large Language Models for Bengali Math Word Problem Solving with Chain of Thought Reasoning
  4. 2026When a Name Is Not a Name: A Benchmark Dataset and Distilled Reasoning for Culturally Entangled Bangla Homographs in Low-Resource LLMs
  5. 2024Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs
  6. 2025Culturally-Grounded Chain-of-Thought (CG-CoT):Enhancing LLM Performance on Culturally-Specific Tasks in Low-Resource Languages
  7. 2025Structured Reasoning with Tree-of-Thoughts for Bengali Math Word Problems
  8. 2026When Meaning Isn't Literal: Exploring Idiomatic Meaning Across Languages and Modalities