Reveal-Bangla: A Dataset for Cross-Lingual Multi-Step Reasoning Evaluation
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Language models have demonstrated remarkable performance on complex multi-step reasoning tasks. However, their evaluation has been predominantly confined to high-resource languages such as English. In this paper, we introduce a manually translated Bangla multi-step reasoning dataset derived from the English Reveal dataset, featuring both binary and non-binary question types. We conduct a controlled evaluation of English-centric and Bangla-centric multilingual small language models on the original dataset and our translated version to compare their ability to exploit relevant reasoning steps to produce correct answers. Our results show that, in comparable settings, reasoning context is beneficial for more challenging non-binary questions, but models struggle to employ relevant Bangla reasoning steps effectively. We conclude by exploring how reasoning steps contribute to models' predictions, highlighting different trends across models and languages.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 12 of its 38 references that are in the map. First posted on arXiv 12 Aug 2025.
Builds on
12 of 38 references.
Closest ancestors · 1
Same direction, newest first
- 2025When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracy28 May
Other references · 11
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2022UL2: Unifying Language Learning Paradigmscited by 295
- 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
- 2024The Impact of Reasoning Step Length on Large Language Modelscited by 153
- 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chainscited by 65
- 2021How Much Coffee Was Consumed During EMNLP 2019? Fermi Problems: A New Reasoning Challenge for AIcited by 22
- 2024BanglaLlama: LLaMA for Bangla Languagecited by 5
- 2024BanglaQuAD: A Bengali Open-domain Question Answering Datasetcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMs
- 2026Rethinking the Multilingual Reasoning Gap with Layer Swap
- 2023Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention
- 2025Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
- 2025Can LLMs Solve My Grandma's Riddle? Evaluating Multilingual Large Language Models on Reasoning Traditional Bangla Tricky Riddles
- 2025A Survey of Multilingual Reasoning in Language Models
- 2024BEnQA: A Question Answering and Reasoning Benchmark for Bengali and English
- 2026Omanic: Towards Step-wise Evaluation of Multi-hop Reasoning in Large Language Models