ks-pret-5m: a 5 million word, 12 million token kashmiri pretraining dataset
Not on the map. Why?
Abstract (arXiv)
We present KS-PRET-5M, the largest publicly available pretraining dataset for the Kashmiri language, comprising 5,090,244 (5.09M) words, 27,692,959 (27.6M) characters, and a vocabulary of 295,433 (295.4K) unique word types. We assembled the dataset from two source classes: digitized archival and literary material, encompassing literature, news, biographies, novels, poetry, religious scholarship, and academic writing, recovered from the proprietary InPage desktop-publishing format using the converter of Malik~citemalik2024inpage, and Unicode-native text collected from Kashmiri-language web sources. All text was processed through an eleven-stage cleaning pipeline that achieves a mean Kashmiri script ratio of 0.9965, reducing Devanagari contamination to 146 characters across the full dataset. We tokenized the dataset empirically using google/muril-base-cased, yielding a subword ratio of 2.383 tokens per word and a total of approximately 12.13 million subword tokens, substantially higher than prior estimates derived from non-Kashmiri Perso-Arabic analogues. KS-PRET-5M is released as a single continuous text stream under CC~BY~4.0 to support language model pretraining, tokenizer training, and computational linguistic research for Kashmiri.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 11 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
6 of 11 references.
References · 6
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
- 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
- 2021MuRIL: Multilingual Representations for Indian Languagescited by 179
Built on by
Citing papers per year: 2026: 1.
1 paper.