mlsanityData to 14 Oct 2026

Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language

arXiv 2606.16383 ↗First posted 15 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

While the NLP landscape is dominated by multi-billion parameter architectures, their deployment in low-resource, non-Latin scripts remains computationally prohibitive for edge configurations, mobile systems, and decentralized local hardware. This paper presents bangla-smollm-135m, a highly compact 135-million parameter decoder-only foundational model engineered explicitly for high-efficiency language modeling in the Bangla script. By leveraging a deterministic intersect-and-append token merging strategy between TituLLMs and SmolLM2-135M, the model overcomes subword script fragmentation without destabilizing early pretrained parameter states. In zero-shot multi-task benchmark evaluations (PIQA_bn, OpenBookQA_bn, CommonsenseQA_bn, and Bangla_MMLU), bangla-smollm-135m matches or outperforms models twice its size (Gemma-3-270m) and achieves parity with models in the 1B parameter tier. The model is available at rnnandi/bangla-smollm-135m

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

12 of 22 references.

References · 12

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  6. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 347
  7. 2021BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Banglacited by 125
  8. 2020Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translationcited by 53
  9. 2022BanglaNLG and BanglaT5: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Banglacited by 42
  10. 2025TigerLLM - A Family of Bangla Large Language Modelscited by 16
  11. 2025The Token Tax: Systematic Bias in Multilingual Tokenizationcited by 14
  12. 2026Bengali Text Classification: An Evaluation of Large Language Model Approachescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs
  2. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
  3. 2026Quantization Effects on Bangla Language Understanding in Large Language Models: A Systematic Evaluation
  4. 2025From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs
  5. 2025PyBangla at BLP-2025 Task 2: Enhancing Bangla-to-Python Code Generation with Iterative Self-Correction and Multilingual Agents
  6. 2024ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency
  7. 2024Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models
  8. 2024A Thorough Examination of Decoding Methods in the Era of LLMs