mlsanityData to 14 Oct 2026

ks-pret-5m: a 5 million word, 12 million token kashmiri pretraining dataset

arXiv 2604.11066 ↗First posted 13 Apr 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We present KS-PRET-5M, the largest publicly available pretraining dataset for the Kashmiri language, comprising 5,090,244 (5.09M) words, 27,692,959 (27.6M) characters, and a vocabulary of 295,433 (295.4K) unique word types. We assembled the dataset from two source classes: digitized archival and literary material, encompassing literature, news, biographies, novels, poetry, religious scholarship, and academic writing, recovered from the proprietary InPage desktop-publishing format using the converter of Malik~citemalik2024inpage, and Unicode-native text collected from Kashmiri-language web sources. All text was processed through an eleven-stage cleaning pipeline that achieves a mean Kashmiri script ratio of 0.9965, reducing Devanagari contamination to 146 characters across the full dataset. We tokenized the dataset empirically using google/muril-base-cased, yielding a subword ratio of 2.383 tokens per word and a total of approximately 12.13 million subword tokens, substantially higher than prior estimates derived from non-Kashmiri Perso-Arabic analogues. KS-PRET-5M is released as a single continuous text stream under CC~BY~4.0 to support language model pretraining, tokenizer training, and computational linguistic research for Kashmiri.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 11 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

6 of 11 references.

References · 6

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  4. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  6. 2021MuRIL: Multilingual Representations for Indian Languagescited by 179

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.