# TigerLLM - A Family of Bangla Large Language Models

arXiv: https://arxiv.org/abs/2503.10995 · HTML: https://mlsanity.com/paper/2503.10995/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Orca 2: Teaching Small Language Models How to Reason](https://mlsanity.com/paper/2311.11045/) (2023)
- [A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://mlsanity.com/paper/2006.06202/) (2020)
- [BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla](https://mlsanity.com/paper/2101.00204/) (2021)
- [Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries](https://mlsanity.com/paper/2310.13132/) (2023)
- [Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages](https://mlsanity.com/paper/2410.16153/) (2024)
- [mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation](https://mlsanity.com/paper/2410.15037/) (2024)
- [Large Language Models in Computer Science Education: A Systematic Literature Review](https://mlsanity.com/paper/2410.16349/) (2024)
- [Vacaspati: A Diverse Corpus of Bangla Literature](https://mlsanity.com/paper/2307.05083/) (2023)
- [BEnQA: A Question Answering and Reasoning Benchmark for Bengali and English](https://mlsanity.com/paper/2403.10900/) (2024)
- [BanglaQuAD: A Bengali Open-domain Question Answering Dataset](https://mlsanity.com/paper/2410.10229/) (2024)
- [MojoBench: Language Modeling and Benchmarks for Mojo](https://mlsanity.com/paper/2410.17736/) (2024)
- [What we can learn from TikTok through its Research API](https://mlsanity.com/paper/2402.13855/) (2024)
- [Code LLMs: A Taxonomy-based Survey](https://mlsanity.com/paper/2412.08291/) (2024)

## Built on by (most cited first, 16 of 16)

- [BnMMLU: Measuring Massive Multitask Language Understanding in Bengali](https://mlsanity.com/paper/2505.18951/) (2025)
- [GanitLLM: Difficulty-Aware Bengali Mathematical Reasoning through Curriculum-GRPO](https://mlsanity.com/paper/2601.06767/) (2026)
- [BLUCK: A Benchmark Dataset for Bengali Linguistic Understanding and Cultural Knowledge](https://mlsanity.com/paper/2505.21092/) (2025)
- [NCTB-QA: A Large-Scale Bangla Educational Question Answering Dataset and Benchmarking Performance](https://mlsanity.com/paper/2603.05462/) (2026)
- [BengaliMoralBench: A Benchmark for Auditing Moral Reasoning in Large Language Models within Bengali Language and Culture](https://mlsanity.com/paper/2511.03180/) (2025)
- [BenHalluEval: A Multi-Task Hallucination Evaluation Framework for Large Language Models on Bengali](https://mlsanity.com/paper/2605.31483/) (2026)
- [Measuring the Tokenization Premium: A Cost Audit for Underserved Language Communities](https://mlsanity.com/paper/2608.09046/) (2026)
- [Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms](https://mlsanity.com/paper/2608.22335/) (2026)
- [Retriv at BLP-2025 Task 2: Test-Driven Feedback-Guided Framework for Bangla-to-Python Code Generation](https://mlsanity.com/paper/2511.07382/) (2025)
- [Accelerating Bangla NLP Tasks with Automatic Mixed Precision: Resource-Efficient Training Preserving Model Efficacy](https://mlsanity.com/paper/2512.00829/) (2025)
- [Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language](https://mlsanity.com/paper/2606.16383/) (2026)
- [Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms](https://mlsanity.com/paper/2607.21780/) (2026)
- [Where Does Retrieval Fail? Evaluating RAG Architectures for Agricultural Advisory](https://mlsanity.com/paper/2608.14886/) (2026)
- [TRACE-BN: Transferring Bangla-English Tutoring Behavior to a Sub-1B Offline Language Model](https://mlsanity.com/paper/2608.15223/) (2026)
- [Evaluating Communicative Success in Machine-Translated Conversation](https://mlsanity.com/paper/2609.19885/) (2026)
- [BANGLAPLAN: A Benchmark for Constraint-Based Planning and Decision-Making in Bangla](https://mlsanity.com/paper/2610.19704/) (2026)

## Abstract (arXiv)

The development of Large Language Models (LLMs) remains heavily skewed towards English and a few other high-resource languages. This linguistic disparity is particularly evident for Bangla - the 5th most spoken language. A few initiatives attempted to create open-source Bangla LLMs with performance still behind high-resource languages and limited reproducibility. To address this gap, we introduce TigerLLM - a family of Bangla LLMs. Our results demonstrate that these models surpass all open-source alternatives and also outperform larger proprietary models like GPT3.5 across standard benchmarks, establishing TigerLLM as the new baseline for future Bangla language modeling.