mlsanityData to 14 Oct 2026

SHAKTI: A 2.5 Billion Parameter Small Language Model Optimized for Edge AI and Low-Resource Environments

arXiv 2410.11331 ↗First posted 15 Oct 2024cs.CL cs.CV cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

We introduce Shakti, a 2.5 billion parameter language model specifically optimized for resource-constrained environments such as edge devices, including smartphones, wearables, and IoT systems. Shakti combines high-performance NLP with optimized efficiency and precision, making it ideal for real-time AI applications where computational resources and memory are limited. With support for vernacular languages and domain-specific tasks, Shakti excels in industries such as healthcare, finance, and customer service. Benchmark evaluations demonstrate that Shakti performs competitively against larger models while maintaining low latency and on-device efficiency, positioning it as a leading solution for edge AI.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 12 of its 15 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

12 of 15 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  8. 2020GLU Variants Improve Transformercited by 1,804
  9. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  10. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  11. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  12. 2021Systematic Generalization with Edge Transformerscited by 38

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inferencecited by 2
  2. 2026Little Brains, Big Feats: Exploring Compact Language Modelscited by 0